-- Logs begin at Tue 2022-07-26 12:03:51 UTC, end at Tue 2022-07-26 13:07:55 UTC. -- Jul 26 12:36:55.586598 dv-cinder-850939-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 26 12:36:55.597080 dv-cinder-850939-2-iscsi devstack@keystone.service[91738]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 26 12:36:55.599123 dv-cinder-850939-2-iscsi devstack@keystone.service[91738]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Jul 26 12:36:55 2022] *** Jul 26 12:36:55.599123 dv-cinder-850939-2-iscsi devstack@keystone.service[91738]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 26 12:36:55.599123 dv-cinder-850939-2-iscsi devstack@keystone.service[91738]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 26 12:36:55.599123 dv-cinder-850939-2-iscsi devstack@keystone.service[91738]: nodename: dv-cinder-850939-2-iscsi Jul 26 12:36:55.599123 dv-cinder-850939-2-iscsi devstack@keystone.service[91738]: machine: x86_64 Jul 26 12:36:55.599123 dv-cinder-850939-2-iscsi devstack@keystone.service[91738]: clock source: unix Jul 26 12:36:55.599123 dv-cinder-850939-2-iscsi devstack@keystone.service[91738]: pcre jit disabled Jul 26 12:36:55.599123 dv-cinder-850939-2-iscsi devstack@keystone.service[91738]: detected number of CPU cores: 4 Jul 26 12:36:55.599123 dv-cinder-850939-2-iscsi devstack@keystone.service[91738]: current working directory: / Jul 26 12:36:55.599123 dv-cinder-850939-2-iscsi devstack@keystone.service[91738]: detected binary path: /usr/bin/uwsgi-core Jul 26 12:36:55.599689 dv-cinder-850939-2-iscsi devstack@keystone.service[91738]: your processes number limit is 47680 Jul 26 12:36:55.599689 dv-cinder-850939-2-iscsi devstack@keystone.service[91738]: your memory page size is 4096 bytes Jul 26 12:36:55.599689 dv-cinder-850939-2-iscsi devstack@keystone.service[91738]: detected max file descriptor number: 2048 Jul 26 12:36:55.599689 dv-cinder-850939-2-iscsi devstack@keystone.service[91738]: lock engine: pthread robust mutexes Jul 26 12:36:55.599689 dv-cinder-850939-2-iscsi devstack@keystone.service[91738]: thunder lock: enabled Jul 26 12:36:55.599689 dv-cinder-850939-2-iscsi devstack@keystone.service[91738]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 26 12:36:55.599689 dv-cinder-850939-2-iscsi devstack@keystone.service[91738]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 26 12:36:55.638519 dv-cinder-850939-2-iscsi devstack@keystone.service[91738]: Python main interpreter initialized at 0x561aeee669c0 Jul 26 12:36:55.638519 dv-cinder-850939-2-iscsi devstack@keystone.service[91738]: python threads support enabled Jul 26 12:36:55.638519 dv-cinder-850939-2-iscsi devstack@keystone.service[91738]: your server socket listen backlog is limited to 100 connections Jul 26 12:36:55.638519 dv-cinder-850939-2-iscsi devstack@keystone.service[91738]: your mercy for graceful operations on workers is 90 seconds Jul 26 12:36:55.638861 dv-cinder-850939-2-iscsi devstack@keystone.service[91738]: mapped 403077 bytes (393 KB) for 2 cores Jul 26 12:36:55.638937 dv-cinder-850939-2-iscsi devstack@keystone.service[91738]: *** Operational MODE: preforking *** Jul 26 12:36:55.639204 dv-cinder-850939-2-iscsi devstack@keystone.service[91738]: *** uWSGI is running in multiple interpreter mode *** Jul 26 12:36:55.639204 dv-cinder-850939-2-iscsi devstack@keystone.service[91738]: spawned uWSGI master process (pid: 91738) Jul 26 12:36:55.639404 dv-cinder-850939-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 26 12:36:55.643941 dv-cinder-850939-2-iscsi devstack@keystone.service[91738]: spawned uWSGI worker 1 (pid: 91748, cores: 1) Jul 26 12:36:55.643941 dv-cinder-850939-2-iscsi devstack@keystone.service[91738]: spawned uWSGI worker 2 (pid: 91749, cores: 1) Jul 26 12:36:55.643941 dv-cinder-850939-2-iscsi devstack@keystone.service[91738]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 26 12:37:01.180296 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 26 12:37:01.180296 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: from cryptography.utils import int_from_bytes Jul 26 12:37:01.180296 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 26 12:37:01.180296 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: from cryptography.utils import int_from_bytes Jul 26 12:37:01.181675 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 26 12:37:01.181675 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: from cryptography.utils import int_from_bytes Jul 26 12:37:01.181675 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 26 12:37:01.181675 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: from cryptography.utils import int_from_bytes Jul 26 12:37:01.230157 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 26 12:37:01.230358 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 26 12:37:01.230551 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] command line args: [] {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 26 12:37:01.230808 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 26 12:37:01.230898 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 26 12:37:01.230980 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ================================================================================ {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 26 12:37:01.231057 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 26 12:37:01.231194 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] command line args: [] {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 26 12:37:01.231280 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] admin_token = **** {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.231449 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] config_dir = [] {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.231534 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 26 12:37:01.231617 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ================================================================================ {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 26 12:37:01.231720 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.231827 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] config_source = [] {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.231913 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] admin_token = **** {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.232011 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] debug = True {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.232139 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] config_dir = [] {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.232224 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.232755 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.232755 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] config_source = [] {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.232958 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.233064 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] debug = True {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.233165 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] insecure_debug = False {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.233295 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.233427 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.233525 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.233629 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.233733 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] insecure_debug = False {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.233846 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] list_limit = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.233943 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.234043 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] log_config_append = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.234145 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.234257 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.234257 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] log_dir = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.234455 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] list_limit = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.234536 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] log_file = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.234655 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] log_config_append = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.234655 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.234852 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.234928 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] log_dir = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.235044 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.235125 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] log_file = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.235198 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.235320 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.235407 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.235487 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.235602 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.235725 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.235801 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.235919 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.236004 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.236084 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.236202 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.236278 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.236395 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.236476 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.236549 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.236663 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.236745 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.236822 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.236938 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.237023 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.237095 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.237210 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.237285 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] notification_format = cadf {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.237419 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.237506 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.237586 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.237709 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] notification_format = cadf {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.237837 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.237945 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.238078 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] publish_errors = False {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.238240 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.238695 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.238818 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.238902 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] publish_errors = False {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.238978 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.239107 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.239195 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.239341 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.239433 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.239519 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.239601 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] standard_threads = False {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.239779 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.239859 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] strict_password_check = False {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.240003 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.240085 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.240231 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] use_eventlog = False {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.240308 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] standard_threads = False {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.240409 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] use_journal = False {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.240495 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] strict_password_check = False {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.240574 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] use_json = False {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.240693 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.240772 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] use_stderr = False {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.240900 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] use_eventlog = False {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.241047 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] use_syslog = False {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.241177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] use_journal = False {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.241314 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] watch_log_file = False {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.241444 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] use_json = False {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.241572 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.241705 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] use_stderr = False {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.241705 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] use_syslog = False {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.241955 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.242090 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] watch_log_file = False {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 12:37:01.242243 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.242371 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.242499 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.242626 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.242752 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.242887 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.243503 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.243631 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.243743 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.243816 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.243885 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.243963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.244040 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.244116 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.244186 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.244261 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.244331 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.244419 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.244491 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.244666 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.244746 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.244917 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.245000 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.245158 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.245238 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.245402 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.245485 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.245638 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.245717 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.245868 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.245941 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.246118 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.246205 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.246356 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.246436 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.246592 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.246676 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.246845 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.246923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.247090 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.247167 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.247329 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.247409 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.247580 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.247704 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.247837 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.247954 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.248065 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.248170 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.248273 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.248371 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.248511 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.248629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.248743 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.248848 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.248994 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.249101 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.249226 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.249341 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.249446 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.249549 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.249691 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.249771 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.249935 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.250022 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.250169 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.250247 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.250415 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.250492 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.250644 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.250730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.250881 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.250966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.251115 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.251193 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.251345 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.251500 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.251574 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.251685 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.251818 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.251899 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.252049 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.252136 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.252282 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.252359 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.252519 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.252597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.252745 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.252831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.252982 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.253068 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.253362 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.253493 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.253600 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.253707 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.253806 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.253900 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.253900 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.254078 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.254173 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.254277 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.254394 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.254511 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.254625 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.254740 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.254888 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.255020 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.255138 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.255254 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.255369 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.255484 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.255599 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.255762 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.255877 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.255994 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.256110 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.256216 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.256339 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.256456 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.256584 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.256711 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.256839 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.256957 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.257062 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.257178 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.257308 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.257427 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.257550 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.257665 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.257781 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.257904 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.258027 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.258152 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.258269 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.258385 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.258501 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.258649 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.259300 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.259423 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.259540 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.259704 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.259824 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.259940 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.260069 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.260193 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.260316 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] cache.enabled = True {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.260441 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.260558 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.260675 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.260802 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.260920 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.261035 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.261151 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] cache.enabled = True {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.261272 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.261389 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.261515 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.261633 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.261748 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.261871 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.261994 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.262111 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.262235 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.262352 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.262470 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.263089 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.263222 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.263341 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.263466 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.263586 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.263722 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.263840 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.263969 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.264089 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.264284 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.264405 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.264521 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.264638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.264754 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.264869 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.264996 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.265122 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.265253 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.265253 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.265473 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.265589 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.265705 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.265820 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.265948 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.266066 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.266188 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.266305 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.266428 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.266428 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.266654 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.266773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.266888 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.267004 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.267119 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.267243 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.267361 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.267477 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.267593 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.267755 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.267873 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] auth.external = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.267873 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] auth.mapped = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.268105 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.268105 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.268335 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.268454 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.268569 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.268693 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] auth.external = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.268838 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] auth.password = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.268968 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] auth.mapped = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.269095 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] auth.token = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.269095 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.269314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.269431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] catalog.caching = True {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.269548 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.269673 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.269791 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] auth.password = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.269908 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.270024 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] auth.token = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.270147 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.270265 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.270391 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.270509 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] catalog.caching = True {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.270626 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.270752 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.270880 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] credential.caching = True {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.270998 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.271123 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] credential.driver = sql {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.271241 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.271357 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.271475 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.271591 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.271723 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.271853 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.271973 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] credential.caching = True {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.272090 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.272206 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] credential.driver = sql {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.272323 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.272440 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.272569 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.272687 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.272803 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.272920 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.273044 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.273162 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.273290 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.273408 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.273524 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.273640 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.273756 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.273872 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.274001 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.274119 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.274264 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.274381 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.274497 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.274614 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.274744 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] federation.caching = True {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.274863 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.274980 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.275098 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.275214 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] federation.driver = sql {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.275330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.275455 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.275573 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.275716 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.275844 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] federation.caching = True {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.275961 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.276078 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.276209 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.276328 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] federation.driver = sql {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.276512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.276631 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.276748 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.276864 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.276989 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.277107 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.277226 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.277342 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.277458 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.277574 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.277700 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] identity.caching = True {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.277817 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.277934 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.278050 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.278166 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.278363 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.278485 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.278602 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.278725 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.278842 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] identity.caching = True {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.278958 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] identity.driver = sql {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.279082 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.279200 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.279318 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.279434 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.279549 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.279680 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.279815 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.279935 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.280042 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] identity.driver = sql {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.280158 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.280275 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.280392 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.280517 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.280634 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.280634 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.280859 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.280975 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.281092 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.281219 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.281337 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.281337 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.281557 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.281673 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.281789 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.281913 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.282031 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.282148 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.282264 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.282264 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.282490 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.282616 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.282733 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.282850 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.282967 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.283084 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.283200 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.283308 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.283425 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.283553 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.283714 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.283833 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.283949 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.284079 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.284198 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.284314 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.284431 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.284549 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.284666 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.284791 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.284910 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.285026 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.285143 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.285259 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.285376 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.285501 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.285619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.285736 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.285853 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.285969 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.286094 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.286221 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.password = **** {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.286345 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.286463 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.286579 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.286697 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.286813 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.286938 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.287056 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.287172 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.287288 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.287405 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.287521 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.password = **** {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.287664 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.287785 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.287902 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.288018 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.288134 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.288250 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.288379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.288498 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.288615 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.288739 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.288857 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.288980 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.289106 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.289224 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.289348 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.289466 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.289582 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.289705 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.289903 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.user = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.290031 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.290150 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.290267 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.290383 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.290508 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.290626 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.290743 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.290866 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.290983 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.291099 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.291224 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.user = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.291343 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.291459 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.291576 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.291576 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.291846 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.291977 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.292096 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.292213 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.292329 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.292444 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.292561 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.292685 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.292803 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.292921 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.293037 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.293154 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.293269 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.293394 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.293512 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.293512 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.293731 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.293847 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.293964 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.294089 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.294206 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.294323 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.294439 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.294555 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.294660 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.294784 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.294902 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.295019 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.295135 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.295251 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.295367 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.295493 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.295610 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.295745 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.295863 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.295979 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.296096 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.296224 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.296343 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] policy.driver = sql {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.296459 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.296574 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.296574 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.296791 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.296915 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.297031 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.297148 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] receipt.caching = True {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.297264 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.297379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.297496 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.297620 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.297737 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] policy.driver = sql {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.297852 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.297968 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.298083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.298201 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.298325 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.298443 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.298558 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] resource.caching = True {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.298681 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] receipt.caching = True {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.298798 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.298915 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.299039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] resource.driver = sql {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.299157 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.299274 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.299389 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.299505 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.299620 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.299782 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.299901 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.300017 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] revoke.caching = True {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.300132 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] resource.caching = True {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.300250 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.300365 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.300489 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.300606 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] resource.driver = sql {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.300722 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] role.cache_time = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.300842 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.300958 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] role.caching = True {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.301074 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.301260 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] role.driver = sql {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.301381 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.301497 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] role.list_limit = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.301613 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] revoke.caching = True {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.301729 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.301844 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.301973 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.302092 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.302207 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.302324 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] role.cache_time = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.302440 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.302556 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] role.caching = True {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.302680 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.302798 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] role.driver = sql {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.302914 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.303030 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] role.list_limit = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.303145 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.303269 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.303388 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.303503 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.303620 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.303756 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.303873 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.304001 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.304120 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.304236 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.304352 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.304469 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.304585 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.304710 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.304828 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.304944 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.305068 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.305184 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.305300 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.305907 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.306030 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.306147 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.306263 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.306379 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.306494 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.306619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.306736 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.306736 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.306963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.306963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.307181 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.307181 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.307410 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.307526 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.307748 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.307870 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.307987 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.308116 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.308235 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.308352 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.308469 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.308586 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.308712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.308836 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.308953 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.309070 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.309186 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.309302 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.309426 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.309544 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] token.cache_time = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.309661 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.309785 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] token.caching = True {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.309901 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.310018 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.310142 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.310260 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] token.provider = fernet {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.310376 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.310492 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.310607 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.310724 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.310853 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.310972 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.311088 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.311204 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.311320 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] token.cache_time = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.311436 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.311560 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] token.caching = True {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.311691 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.311811 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.311937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] trust.driver = sql {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.312056 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] token.provider = fernet {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.312172 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.312297 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.312415 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.312532 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.312648 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.312764 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.312882 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.313007 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.313125 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.313242 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.313357 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.313473 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.313589 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.313714 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] trust.driver = sql {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.313832 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.313950 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.314066 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] database.connection = **** {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.314191 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.314317 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.314435 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.314552 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.314677 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.314795 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.314914 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.315030 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.315148 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.315264 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.315390 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.315508 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.315624 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.315799 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.315799 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.316075 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] database.connection = **** {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.316203 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.316333 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.316451 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.316568 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.316685 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.316801 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.316917 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.317043 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.317162 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.317279 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.317395 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.317512 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.317628 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.317753 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.317871 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.317988 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.318105 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.318221 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.318337 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.318462 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.318580 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 26 12:37:01.318697 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.318697 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.318697 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.318697 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.318697 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.318697 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.318697 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.318697 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 12:37:01.318697 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 26 12:37:01.344840 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=91748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.345068 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=91748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.345295 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=91748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.345929 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=91749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.346166 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=91749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.346390 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=91749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.353645 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=91748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.354802 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=91749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.384365 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=91748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.384594 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=91748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.385573 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=91749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.385814 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=91749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.436927 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=91749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.437073 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=91748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.487337 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=91748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.487500 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=91749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.487695 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=91748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.487782 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=91749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.552252 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.552471 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=91748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.552692 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=91748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.552881 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.552969 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=91748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.553107 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=91748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.553191 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=91749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.553320 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=91748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.553405 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=91749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.553539 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=91749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.553760 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=91749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.553980 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=91749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.576475 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.576709 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=91748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.576895 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.576999 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=91748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.577127 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=91749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.577213 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=91748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.577352 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=91748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.577447 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=91749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.577566 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=91748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.577656 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=91749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.577776 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.577858 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=91749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.577993 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=91749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.578214 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.588870 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=91748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.589264 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=91749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.601589 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.601811 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=91748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.601963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.602052 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=91748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.602201 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=91749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.602768 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=91748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.602950 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=91749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.603055 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=91748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.603159 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=91748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.603242 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=91749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.603368 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.603452 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=91749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.603582 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=91749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.603830 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 12:37:01.620524 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.620730 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 26 12:37:01.621061 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.621220 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.621318 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 26 12:37:01.621433 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 26 12:37:01.621516 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.621754 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.621836 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.621959 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 26 12:37:01.622044 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.622231 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.622313 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.622498 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.622755 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.623009 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.631848 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.632122 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.632383 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.632655 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.632865 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.633144 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.633415 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.634237 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.640066 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=91748) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 12:37:01.641207 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=91749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 12:37:01.644740 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=91748) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 12:37:01.645015 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.645293 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.645570 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.645847 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.645973 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=91749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 12:37:01.646104 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.646258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.646369 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.646538 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.646629 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.646816 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.646898 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.647092 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.647174 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.647350 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.647441 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.647616 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.647913 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.648181 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.648452 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.648717 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.670781 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.672319 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=91748) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 12:37:01.672485 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.672613 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.674254 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=91749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 12:37:01.674535 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.677370 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=91748) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 12:37:01.677669 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.678015 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.679599 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=91749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 12:37:01.679989 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.680295 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.685510 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=91748) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 12:37:01.685791 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.687757 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=91749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 12:37:01.687979 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.689927 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=91748) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 12:37:01.690312 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.690599 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.690899 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.691170 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.691448 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.691752 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.692041 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.692317 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=91749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 12:37:01.692604 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.692885 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.693166 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.693450 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.693736 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.694020 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.694297 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.707797 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.708071 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.710232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.710506 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.712047 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=91748) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 12:37:01.714521 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=91749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 12:37:01.715400 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.715510 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.717701 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.718031 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.719826 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=91748) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 12:37:01.722403 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=91749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 12:37:01.722986 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=91748) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 12:37:01.725652 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=91749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 12:37:01.726172 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.726461 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.726740 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.727021 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.727838 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.728141 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.729322 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.729623 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.729907 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.730209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.730484 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.730780 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.744835 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=91748) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 12:37:01.745115 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.745378 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.745634 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.747573 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=91749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 12:37:01.747882 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.748154 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.748416 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.752475 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.753994 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.754267 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.755298 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.756869 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.757084 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=91748) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 12:37:01.757167 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.757363 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.757659 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.757944 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.758226 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.760003 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=91749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 12:37:01.760286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.760588 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.760870 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.761157 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.769646 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=91748) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 12:37:01.769924 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.770202 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.770485 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.770759 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.771026 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.771292 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.772578 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=91749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 12:37:01.772861 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.773141 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.773422 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.773708 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.773984 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.774287 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.787732 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=91748) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 12:37:01.791520 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=91748) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 12:37:01.792806 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=91749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 12:37:01.794816 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.796574 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=91749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 12:37:01.797736 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.799298 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=91748) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 12:37:01.799582 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.799882 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.799998 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.802755 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.804403 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=91749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 12:37:01.804689 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.804975 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.806943 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.808560 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=91748) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 12:37:01.811467 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.811780 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.812047 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.812134 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.812324 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.813660 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=91749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 12:37:01.816624 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.816915 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.817212 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.817490 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.820298 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=91748) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 12:37:01.820583 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.820855 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.825593 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=91749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 12:37:01.825888 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.826171 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.827712 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=91748) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 12:37:01.827992 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.828263 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.828540 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.828806 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.829079 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.829353 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.829629 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.829995 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.830270 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.830543 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.830800 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.831076 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.831339 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=91748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.833052 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=91749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 12:37:01.833335 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.833619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.833909 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.834207 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.834479 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.834752 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.835021 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.835393 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.835697 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.835982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.836259 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.836539 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.837331 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=91749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 12:37:01.866001 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x561aeee669c0 pid: 91748 (default app) Jul 26 12:37:01.869876 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7388f-fbdb-4047-87fe-7e2fb009e5c4 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:01.870162 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7388f-fbdb-4047-87fe-7e2fb009e5c4 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:01.870433 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7388f-fbdb-4047-87fe-7e2fb009e5c4 None None] PATH_INFO: `/v3` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:01.871302 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x561aeee669c0 pid: 91749 (default app) Jul 26 12:37:01.874740 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1/1] 10.222.0.20 () {50 vars in 791 bytes} [Tue Jul 26 12:37:01 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:09.547446 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd666931-59c2-46e7-8ce3-2a4194c9eb57 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:09.548051 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd666931-59c2-46e7-8ce3-2a4194c9eb57 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:09.548051 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd666931-59c2-46e7-8ce3-2a4194c9eb57 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:09.552502 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1/2] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 12:37:09.557815 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1feb02-98ac-4123-8aaa-c7db8fdc764d None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:09.558269 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1feb02-98ac-4123-8aaa-c7db8fdc764d None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:09.558355 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1feb02-98ac-4123-8aaa-c7db8fdc764d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:09.628792 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: INFO dbcounter [None req-df1feb02-98ac-4123-8aaa-c7db8fdc764d None None] Registered counter for database keystone Jul 26 12:37:09.650080 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG oslo_db.sqlalchemy.engines [None req-df1feb02-98ac-4123-8aaa-c7db8fdc764d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=91748) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 26 12:37:09.656315 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writer thread running {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 26 12:37:09.839823 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG passlib.handlers.bcrypt [None req-df1feb02-98ac-4123-8aaa-c7db8fdc764d None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=91748) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 26 12:37:09.840152 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG passlib.handlers.bcrypt [None req-df1feb02-98ac-4123-8aaa-c7db8fdc764d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=91748) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 26 12:37:09.896499 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-df1feb02-98ac-4123-8aaa-c7db8fdc764d None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:09.898104 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-df1feb02-98ac-4123-8aaa-c7db8fdc764d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:10.025101 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 2/3] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:09 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 468 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 26 12:37:10.187146 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd666931-59c2-46e7-8ce3-2a4194c9eb57 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:10.256640 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: INFO dbcounter [None req-dd666931-59c2-46e7-8ce3-2a4194c9eb57 None None] Registered counter for database keystone Jul 26 12:37:10.276831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG oslo_db.sqlalchemy.engines [None req-dd666931-59c2-46e7-8ce3-2a4194c9eb57 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=91749) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 26 12:37:10.283095 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writer thread running {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 26 12:37:10.402634 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd666931-59c2-46e7-8ce3-2a4194c9eb57 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:10.406930 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c91fe-6703-45e7-b015-9cc85fabe167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:10.407563 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c91fe-6703-45e7-b015-9cc85fabe167 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:10.407876 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c91fe-6703-45e7-b015-9cc85fabe167 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:10.408159 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c91fe-6703-45e7-b015-9cc85fabe167 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:10.416468 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311c91fe-6703-45e7-b015-9cc85fabe167 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:10.469096 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: WARNING keystone.common.rbac_enforcer.enforcer [None req-311c91fe-6703-45e7-b015-9cc85fabe167 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 26 12:37:10.469430 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311c91fe-6703-45e7-b015-9cc85fabe167 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:10.480531 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 2/4] 10.222.0.20 () {58 vars in 1324 bytes} [Tue Jul 26 12:37:10 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 12:37:10.486519 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1feb02-98ac-4123-8aaa-c7db8fdc764d None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:10.501585 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1feb02-98ac-4123-8aaa-c7db8fdc764d None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:10.506109 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06669ae5-b413-4c40-8516-fdc2fabb25de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:10.507235 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06669ae5-b413-4c40-8516-fdc2fabb25de admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:10.507520 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06669ae5-b413-4c40-8516-fdc2fabb25de admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:10.507825 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06669ae5-b413-4c40-8516-fdc2fabb25de admin admin] PATH_INFO: `/v3/projects/1ca671277c17458082e4383a242d5404` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:10.509842 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06669ae5-b413-4c40-8516-fdc2fabb25de admin admin] RBAC: Authorizing `identity:get_project(project_id=1ca671277c17458082e4383a242d5404)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:10.561112 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: WARNING keystone.common.rbac_enforcer.enforcer [None req-06669ae5-b413-4c40-8516-fdc2fabb25de admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 26 12:37:10.561757 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06669ae5-b413-4c40-8516-fdc2fabb25de admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:10.566272 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 3/5] 10.222.0.20 () {56 vars in 1207 bytes} [Tue Jul 26 12:37:10 2022] GET /identity/v3/projects/1ca671277c17458082e4383a242d5404 => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:37:12.558300 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ab237-aa45-44be-a216-3c3f6e84ab0f None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:12.558844 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ab237-aa45-44be-a216-3c3f6e84ab0f None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:12.558844 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ab237-aa45-44be-a216-3c3f6e84ab0f None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:12.562983 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 3/6] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 12:37:12.568502 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac80ed-6fdf-4a06-8a7c-8276ee5fd472 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:12.568926 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac80ed-6fdf-4a06-8a7c-8276ee5fd472 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:12.569056 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac80ed-6fdf-4a06-8a7c-8276ee5fd472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:12.589305 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-21ac80ed-6fdf-4a06-8a7c-8276ee5fd472 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:12.590803 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-21ac80ed-6fdf-4a06-8a7c-8276ee5fd472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:12.598591 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 4/7] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:12 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 26 12:37:12.620495 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ab237-aa45-44be-a216-3c3f6e84ab0f None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:12.632581 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ab237-aa45-44be-a216-3c3f6e84ab0f None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:12.637389 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3652d49-351e-4b12-92c3-0b7434d4f7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:12.637995 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3652d49-351e-4b12-92c3-0b7434d4f7e7 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:12.638282 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3652d49-351e-4b12-92c3-0b7434d4f7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:12.638563 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3652d49-351e-4b12-92c3-0b7434d4f7e7 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:12.646349 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3652d49-351e-4b12-92c3-0b7434d4f7e7 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:12.648452 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3652d49-351e-4b12-92c3-0b7434d4f7e7 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:12.657326 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 4/8] 10.222.0.20 () {58 vars in 1324 bytes} [Tue Jul 26 12:37:12 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 12:37:12.662759 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac80ed-6fdf-4a06-8a7c-8276ee5fd472 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:12.671889 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac80ed-6fdf-4a06-8a7c-8276ee5fd472 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:12.676342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9a71c4-aece-466a-9e90-bf7b7638e57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:12.676977 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a71c4-aece-466a-9e90-bf7b7638e57d admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:12.677264 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a71c4-aece-466a-9e90-bf7b7638e57d admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:12.677541 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a71c4-aece-466a-9e90-bf7b7638e57d admin admin] PATH_INFO: `/v3/users/26e313f5057b4411a71a6bd67e68cb0e` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:12.678933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9a71c4-aece-466a-9e90-bf7b7638e57d admin admin] RBAC: Authorizing `identity:get_user(user_id=26e313f5057b4411a71a6bd67e68cb0e)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:12.681297 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9a71c4-aece-466a-9e90-bf7b7638e57d admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:12.682852 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 5/9] 10.222.0.20 () {56 vars in 1198 bytes} [Tue Jul 26 12:37:12 2022] GET /identity/v3/users/26e313f5057b4411a71a6bd67e68cb0e => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:14.766021 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c556a-54aa-4840-b389-be936edb8c4c None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:14.766564 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c556a-54aa-4840-b389-be936edb8c4c None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:14.766564 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c556a-54aa-4840-b389-be936edb8c4c None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:14.768090 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 5/10] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:37:14.773440 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff1e99f-5963-4d7b-927e-870a359bd322 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:14.774332 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff1e99f-5963-4d7b-927e-870a359bd322 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:14.774513 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff1e99f-5963-4d7b-927e-870a359bd322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:14.793006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa1a1b-8a7c-4d11-83d3-d42307fffb41 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:14.793298 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa1a1b-8a7c-4d11-83d3-d42307fffb41 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:14.793576 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa1a1b-8a7c-4d11-83d3-d42307fffb41 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:14.794641 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-9ff1e99f-5963-4d7b-927e-870a359bd322 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:14.795036 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 6/11] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 12:37:14.795887 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-9ff1e99f-5963-4d7b-927e-870a359bd322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:14.800265 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd14cdf0-ed40-47d2-ab49-43da13b6ce68 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:14.800552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd14cdf0-ed40-47d2-ab49-43da13b6ce68 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:14.800829 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd14cdf0-ed40-47d2-ab49-43da13b6ce68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:14.843344 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG passlib.handlers.bcrypt [None req-cd14cdf0-ed40-47d2-ab49-43da13b6ce68 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=91749) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 26 12:37:14.843809 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG passlib.handlers.bcrypt [None req-cd14cdf0-ed40-47d2-ab49-43da13b6ce68 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=91749) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 26 12:37:14.878236 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 6/12] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:37:14.890240 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-cd14cdf0-ed40-47d2-ab49-43da13b6ce68 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:14.891955 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-cd14cdf0-ed40-47d2-ab49-43da13b6ce68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:14.901542 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff1e99f-5963-4d7b-927e-870a359bd322 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:14.925011 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:14.994585 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 7/13] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 12:37:15.007191 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff1e99f-5963-4d7b-927e-870a359bd322 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:15.016926 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd14cdf0-ed40-47d2-ab49-43da13b6ce68 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:15.041979 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6a7b7d-af6a-4831-ad95-c30d75c39d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:15.042621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a7b7d-af6a-4831-ad95-c30d75c39d35 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:15.042900 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a7b7d-af6a-4831-ad95-c30d75c39d35 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:15.043173 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a7b7d-af6a-4831-ad95-c30d75c39d35 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:15.054427 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6a7b7d-af6a-4831-ad95-c30d75c39d35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:37:15.063204 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-eb6a7b7d-af6a-4831-ad95-c30d75c39d35 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:37:15.063204 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:37:15.063204 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:37:15.063204 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:37:15.063204 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:37:15.063204 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:37:15.063204 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:37:15.063204 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:37:15.063204 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:37:15.063204 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:37:15.063204 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:37:15.063204 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:37:15.063204 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:37:15.063204 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:37:15.063204 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:37:15.063204 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:37:15.064320 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:37:15.064320 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:37:15.064320 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:37:15.064320 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:37:15.064320 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:37:15.064320 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:37:15.064320 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:37:15.064320 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:37:15.064320 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:37:15.064320 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:37:15.064320 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:37:15.064320 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:37:15.064320 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:37:15.064320 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:37:15.064320 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:37:15.064320 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:37:15.065379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:37:15.065379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:37:15.065379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:37:15.065379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:37:15.065379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:37:15.065379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:37:15.065379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:37:15.065379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:37:15.065379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:37:15.065379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:37:15.065379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:37:15.065379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:37:15.065379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 7/14] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 26 12:37:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:37:15.070326 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6a7b7d-af6a-4831-ad95-c30d75c39d35 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:15.120305 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd14cdf0-ed40-47d2-ab49-43da13b6ce68 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:15.122586 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:15.154794 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:15.159519 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070b0cc3-e588-4b63-9f68-8a78fbf46a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:15.160447 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b0cc3-e588-4b63-9f68-8a78fbf46a96 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:15.160732 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b0cc3-e588-4b63-9f68-8a78fbf46a96 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:15.161011 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b0cc3-e588-4b63-9f68-8a78fbf46a96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:15.175207 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6a7b7d-af6a-4831-ad95-c30d75c39d35 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:15.210610 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d324a4e-0cb2-43e4-bf1a-9b38ecfdf998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:15.211270 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d324a4e-0cb2-43e4-bf1a-9b38ecfdf998 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:15.212083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d324a4e-0cb2-43e4-bf1a-9b38ecfdf998 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:15.212361 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d324a4e-0cb2-43e4-bf1a-9b38ecfdf998 None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:15.213087 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d324a4e-0cb2-43e4-bf1a-9b38ecfdf998 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:15.213394 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d324a4e-0cb2-43e4-bf1a-9b38ecfdf998 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:15.214791 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d324a4e-0cb2-43e4-bf1a-9b38ecfdf998 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:15.226586 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 8/15] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:37:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:37:15.232228 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d324a4e-0cb2-43e4-bf1a-9b38ecfdf998 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:15.234981 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070b0cc3-e588-4b63-9f68-8a78fbf46a96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:15.236519 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070b0cc3-e588-4b63-9f68-8a78fbf46a96 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:15.332815 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 8/16] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:37:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 12:37:15.333980 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d324a4e-0cb2-43e4-bf1a-9b38ecfdf998 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:15.338398 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070b0cc3-e588-4b63-9f68-8a78fbf46a96 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:15.341147 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:15.445107 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070b0cc3-e588-4b63-9f68-8a78fbf46a96 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:15.475351 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e5d97-47b5-4578-bea0-bb3b212af0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:15.476018 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e5d97-47b5-4578-bea0-bb3b212af0e8 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:15.476301 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e5d97-47b5-4578-bea0-bb3b212af0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:15.476574 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e5d97-47b5-4578-bea0-bb3b212af0e8 None admin] PATH_INFO: `/v3/users/26e313f5057b4411a71a6bd67e68cb0e` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:15.477815 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5e5d97-47b5-4578-bea0-bb3b212af0e8 None admin] RBAC: Authorizing `identity:get_user(user_id=26e313f5057b4411a71a6bd67e68cb0e)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:15.479254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5e5d97-47b5-4578-bea0-bb3b212af0e8 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:15.480661 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e500ad-8772-433b-b864-f6d9fabceba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:15.481214 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 9/17] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:37:15 2022] GET /identity/v3/users/26e313f5057b4411a71a6bd67e68cb0e => generated 247 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:37:15.481518 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e500ad-8772-433b-b864-f6d9fabceba6 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:15.481802 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e500ad-8772-433b-b864-f6d9fabceba6 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:15.482079 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e500ad-8772-433b-b864-f6d9fabceba6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:15.486255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e5d97-47b5-4578-bea0-bb3b212af0e8 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:15.501432 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e500ad-8772-433b-b864-f6d9fabceba6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:15.502301 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:15.502923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e500ad-8772-433b-b864-f6d9fabceba6 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:15.521975 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-49e500ad-8772-433b-b864-f6d9fabceba6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:37:15.521975 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:37:15.521975 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:37:15.521975 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:37:15.521975 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:37:15.521975 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:37:15.521975 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:37:15.521975 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:37:15.521975 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:37:15.521975 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:37:15.521975 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:37:15.521975 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:37:15.521975 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:37:15.521975 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:37:15.521975 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:37:15.522977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:37:15.522977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:37:15.522977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:37:15.522977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:37:15.522977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:37:15.522977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:37:15.522977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:37:15.522977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:37:15.522977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:37:15.522977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:37:15.522977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:37:15.522977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:37:15.522977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:37:15.522977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:37:15.522977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:37:15.522977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:37:15.522977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:37:15.523792 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:37:15.523792 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:37:15.523792 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:37:15.523792 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:37:15.523792 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 9/18] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:37:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:37:15.529615 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e500ad-8772-433b-b864-f6d9fabceba6 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:15.586098 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:15.594551 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e5d97-47b5-4578-bea0-bb3b212af0e8 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:15.628445 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9276c0d6-bfd9-4c86-b005-6e2e5970d071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:15.629085 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276c0d6-bfd9-4c86-b005-6e2e5970d071 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:15.629366 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276c0d6-bfd9-4c86-b005-6e2e5970d071 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:15.629639 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276c0d6-bfd9-4c86-b005-6e2e5970d071 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:15.630808 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9276c0d6-bfd9-4c86-b005-6e2e5970d071 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:15.631788 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e500ad-8772-433b-b864-f6d9fabceba6 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:15.632292 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9276c0d6-bfd9-4c86-b005-6e2e5970d071 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:15.634474 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 10/19] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:37:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:37:15.640479 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9276c0d6-bfd9-4c86-b005-6e2e5970d071 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:15.665862 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41947b7b-8a0e-4092-b354-c66e1406149d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:15.666488 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41947b7b-8a0e-4092-b354-c66e1406149d None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:15.666774 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41947b7b-8a0e-4092-b354-c66e1406149d None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:15.667056 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41947b7b-8a0e-4092-b354-c66e1406149d None admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:15.667786 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41947b7b-8a0e-4092-b354-c66e1406149d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:15.668097 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41947b7b-8a0e-4092-b354-c66e1406149d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:15.669488 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41947b7b-8a0e-4092-b354-c66e1406149d None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:15.687350 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 10/20] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:37:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:37:15.724662 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:15.747234 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9276c0d6-bfd9-4c86-b005-6e2e5970d071 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:15.782805 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84957c38-bd09-4463-bb1c-a24fbab2a4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:15.783534 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84957c38-bd09-4463-bb1c-a24fbab2a4d9 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:15.783746 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84957c38-bd09-4463-bb1c-a24fbab2a4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:15.784033 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84957c38-bd09-4463-bb1c-a24fbab2a4d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:15.784844 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84957c38-bd09-4463-bb1c-a24fbab2a4d9 None admin] RBAC: Adding query filter params (role.id=0e78444e43c744668178971af4087ca0, scope.domain.id=default, user.id=26e313f5057b4411a71a6bd67e68cb0e) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:15.785156 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84957c38-bd09-4463-bb1c-a24fbab2a4d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:15.786614 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84957c38-bd09-4463-bb1c-a24fbab2a4d9 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:15.794675 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 11/21] 10.222.0.20 () {56 vars in 1322 bytes} [Tue Jul 26 12:37:15 2022] GET /identity/v3/role_assignments?user.id=26e313f5057b4411a71a6bd67e68cb0e&scope.domain.id=default&role.id=0e78444e43c744668178971af4087ca0 => generated 233 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:18.192083 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14e0a5-33c7-4c6c-a5b8-d296cdbc3156 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:18.192929 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14e0a5-33c7-4c6c-a5b8-d296cdbc3156 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:18.193370 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14e0a5-33c7-4c6c-a5b8-d296cdbc3156 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:18.194866 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 11/22] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:37:18.202447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc0308-9f17-43de-a081-95f16a223cd8 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:18.203250 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc0308-9f17-43de-a081-95f16a223cd8 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:18.203695 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc0308-9f17-43de-a081-95f16a223cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:18.232990 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-89dc0308-9f17-43de-a081-95f16a223cd8 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:18.234419 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-89dc0308-9f17-43de-a081-95f16a223cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:18.291090 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25203a97-272c-4dc8-8622-a9a4b32d3fb8 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:18.291602 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25203a97-272c-4dc8-8622-a9a4b32d3fb8 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:18.292043 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25203a97-272c-4dc8-8622-a9a4b32d3fb8 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:18.293545 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 12/23] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 12:37:18.298992 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eac714-8dc4-4460-a2a2-386d76496d2d None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:18.299442 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eac714-8dc4-4460-a2a2-386d76496d2d None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:18.299967 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eac714-8dc4-4460-a2a2-386d76496d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:18.335268 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-d4eac714-8dc4-4460-a2a2-386d76496d2d None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:18.337575 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-d4eac714-8dc4-4460-a2a2-386d76496d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:18.357014 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 12/24] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:37:18.358535 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3e196-4072-4749-b298-da232eb1acf8 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:18.358953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3e196-4072-4749-b298-da232eb1acf8 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:18.359325 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3e196-4072-4749-b298-da232eb1acf8 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:18.360871 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 13/25] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:37:18.369523 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc0426-b5d2-4788-b33c-a6bc717124fa None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:18.370078 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc0426-b5d2-4788-b33c-a6bc717124fa None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:18.370555 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc0426-b5d2-4788-b33c-a6bc717124fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:18.408673 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-a9cc0426-b5d2-4788-b33c-a6bc717124fa None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:18.409882 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-a9cc0426-b5d2-4788-b33c-a6bc717124fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:18.427106 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 13/26] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:37:18.428006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eac714-8dc4-4460-a2a2-386d76496d2d None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:18.522523 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:18.554167 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 14/27] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 12:37:18.555750 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc0426-b5d2-4788-b33c-a6bc717124fa None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:18.565698 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:18.636102 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eac714-8dc4-4460-a2a2-386d76496d2d None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:18.704096 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359fe061-aca2-4073-bafd-45b603887725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:18.704730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359fe061-aca2-4073-bafd-45b603887725 None admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:18.705011 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359fe061-aca2-4073-bafd-45b603887725 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:18.705287 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359fe061-aca2-4073-bafd-45b603887725 None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:18.706014 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359fe061-aca2-4073-bafd-45b603887725 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:18.707414 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359fe061-aca2-4073-bafd-45b603887725 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:18.746686 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc0426-b5d2-4788-b33c-a6bc717124fa None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:18.830341 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859fe023-9d6a-4860-8e88-2f0fa06fbf21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:18.830959 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859fe023-9d6a-4860-8e88-2f0fa06fbf21 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:18.831245 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859fe023-9d6a-4860-8e88-2f0fa06fbf21 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:18.831522 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859fe023-9d6a-4860-8e88-2f0fa06fbf21 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:18.864977 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859fe023-9d6a-4860-8e88-2f0fa06fbf21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:37:18.871193 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-859fe023-9d6a-4860-8e88-2f0fa06fbf21 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:37:18.871193 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:37:18.871193 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:37:18.871193 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:37:18.871193 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:37:18.871193 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:37:18.871193 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:37:18.871193 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:37:18.871193 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:37:18.871193 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:37:18.871193 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:37:18.871193 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:37:18.871193 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:37:18.871193 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:37:18.871193 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:37:18.871193 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:37:18.872133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:37:18.872133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:37:18.872133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:37:18.872133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:37:18.872133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:37:18.872133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:37:18.872133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:37:18.872133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:37:18.872133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:37:18.872133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:37:18.872133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:37:18.872133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:37:18.872133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:37:18.872133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:37:18.872133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:37:18.872133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:37:18.872877 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:37:18.872877 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:37:18.872877 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:37:18.872877 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:37:18.872877 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:37:18.872877 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:37:18.872877 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:37:18.872877 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:37:18.872877 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:37:18.872877 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:37:18.872877 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:37:18.872877 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:37:18.882185 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 15/28] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 26 12:37:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:37:18.883534 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859fe023-9d6a-4860-8e88-2f0fa06fbf21 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:18.975985 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:18.990104 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 14/29] 10.222.0.20 () {60 vars in 1131 bytes} [Tue Jul 26 12:37:18 2022] POST /identity/v3/roles => generated 219 bytes in 563 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:37:18.991334 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359fe061-aca2-4073-bafd-45b603887725 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:19.057099 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859fe023-9d6a-4860-8e88-2f0fa06fbf21 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:19.126780 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359fe061-aca2-4073-bafd-45b603887725 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:19.138351 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:19.146532 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20833ff9-d5a7-4370-9b85-4ca25ff42bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:19.147986 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20833ff9-d5a7-4370-9b85-4ca25ff42bca None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:19.147986 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20833ff9-d5a7-4370-9b85-4ca25ff42bca None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:19.148283 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20833ff9-d5a7-4370-9b85-4ca25ff42bca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:19.170386 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20833ff9-d5a7-4370-9b85-4ca25ff42bca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:19.172491 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20833ff9-d5a7-4370-9b85-4ca25ff42bca None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:19.173411 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263bd6f4-0a6d-44b2-b37b-b1c4d759afca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:19.174135 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263bd6f4-0a6d-44b2-b37b-b1c4d759afca None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:19.174491 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263bd6f4-0a6d-44b2-b37b-b1c4d759afca None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:19.174830 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263bd6f4-0a6d-44b2-b37b-b1c4d759afca None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:19.175849 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263bd6f4-0a6d-44b2-b37b-b1c4d759afca None admin] RBAC: Adding query filter params (name=admin) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:19.176363 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263bd6f4-0a6d-44b2-b37b-b1c4d759afca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:19.178193 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263bd6f4-0a6d-44b2-b37b-b1c4d759afca None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:19.193580 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 15/30] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:37:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:19.196170 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a18d8c-2be3-481b-ac0d-76fb5910929d None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:19.196686 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a18d8c-2be3-481b-ac0d-76fb5910929d None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:19.196822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-20833ff9-d5a7-4370-9b85-4ca25ff42bca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:37:19.196822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:37:19.196822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:37:19.196822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:37:19.196822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:37:19.196822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:37:19.196822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:37:19.196822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:37:19.196822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:37:19.196822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:37:19.196822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:37:19.196822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:37:19.196822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:37:19.196822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:37:19.196822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:37:19.197881 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:37:19.197881 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:37:19.197881 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:37:19.197881 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:37:19.197881 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:37:19.197881 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:37:19.197881 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:37:19.197881 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:37:19.197881 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:37:19.197881 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:37:19.197881 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:37:19.197881 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:37:19.197881 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:37:19.197881 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:37:19.197881 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:37:19.197881 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:37:19.197881 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:37:19.198895 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:37:19.198895 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:37:19.198895 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:37:19.198895 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:37:19.199141 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a18d8c-2be3-481b-ac0d-76fb5910929d None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:19.200255 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 16/31] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:37:19.201710 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509a103-fa0f-4240-bd96-6d9e193dacc2 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:19.202171 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509a103-fa0f-4240-bd96-6d9e193dacc2 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:19.202607 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509a103-fa0f-4240-bd96-6d9e193dacc2 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:19.204375 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 17/32] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:37:19.205268 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c509a103-fa0f-4240-bd96-6d9e193dacc2 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:19.206772 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 16/33] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:37:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:37:19.208226 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266cc18b-5f9a-43bb-9089-9112cba00451 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:19.208726 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266cc18b-5f9a-43bb-9089-9112cba00451 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:19.209255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266cc18b-5f9a-43bb-9089-9112cba00451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:19.233908 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-266cc18b-5f9a-43bb-9089-9112cba00451 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:19.235341 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-266cc18b-5f9a-43bb-9089-9112cba00451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:19.319441 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 17/34] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:37:19.320947 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266cc18b-5f9a-43bb-9089-9112cba00451 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:19.331852 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c509a103-fa0f-4240-bd96-6d9e193dacc2 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:19.368797 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c6a07b-2ecb-40b3-a1e2-f64c8afc1fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:19.369458 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c6a07b-2ecb-40b3-a1e2-f64c8afc1fa1 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:19.369752 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c6a07b-2ecb-40b3-a1e2-f64c8afc1fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:19.370024 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c6a07b-2ecb-40b3-a1e2-f64c8afc1fa1 None admin] PATH_INFO: `/v3/users/26e313f5057b4411a71a6bd67e68cb0e` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:19.371887 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c6a07b-2ecb-40b3-a1e2-f64c8afc1fa1 None admin] RBAC: Authorizing `identity:get_user(user_id=26e313f5057b4411a71a6bd67e68cb0e)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:19.373514 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c6a07b-2ecb-40b3-a1e2-f64c8afc1fa1 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:19.375294 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 18/35] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:37:19 2022] GET /identity/v3/users/26e313f5057b4411a71a6bd67e68cb0e => generated 247 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:19.376463 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9071a075-4278-410d-adba-47a6de47ff68 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:19.376751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9071a075-4278-410d-adba-47a6de47ff68 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:19.377087 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9071a075-4278-410d-adba-47a6de47ff68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:19.378076 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:19.395748 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-9071a075-4278-410d-adba-47a6de47ff68 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:19.397273 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-9071a075-4278-410d-adba-47a6de47ff68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:19.429536 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:19.440111 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266cc18b-5f9a-43bb-9089-9112cba00451 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:19.472747 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 19/36] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:37:19.473968 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9071a075-4278-410d-adba-47a6de47ff68 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:19.488313 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000eae6a-a13b-450f-b747-ba296677933d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:19.489778 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000eae6a-a13b-450f-b747-ba296677933d None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:19.490266 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000eae6a-a13b-450f-b747-ba296677933d None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:19.491102 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000eae6a-a13b-450f-b747-ba296677933d None admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:19.492353 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000eae6a-a13b-450f-b747-ba296677933d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:19.492883 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000eae6a-a13b-450f-b747-ba296677933d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:19.495241 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000eae6a-a13b-450f-b747-ba296677933d None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:19.515001 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 18/37] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:37:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:19.516226 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000eae6a-a13b-450f-b747-ba296677933d None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:19.591102 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9071a075-4278-410d-adba-47a6de47ff68 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:19.631752 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8efec0-ad33-4046-916d-7165516140e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:19.632569 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8efec0-ad33-4046-916d-7165516140e9 None admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:19.632857 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8efec0-ad33-4046-916d-7165516140e9 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:19.633174 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8efec0-ad33-4046-916d-7165516140e9 None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:19.634365 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8efec0-ad33-4046-916d-7165516140e9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:19.635882 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8efec0-ad33-4046-916d-7165516140e9 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:19.643422 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000eae6a-a13b-450f-b747-ba296677933d None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:19.650621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 20/38] 10.222.0.20 () {60 vars in 1131 bytes} [Tue Jul 26 12:37:19 2022] POST /identity/v3/roles => generated 225 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:37:19.651409 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8efec0-ad33-4046-916d-7165516140e9 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:19.652555 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:19.684002 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405c10d-1f61-4f0c-b20f-aa666f6c81f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:19.684671 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405c10d-1f61-4f0c-b20f-aa666f6c81f8 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:19.684957 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405c10d-1f61-4f0c-b20f-aa666f6c81f8 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:19.685230 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405c10d-1f61-4f0c-b20f-aa666f6c81f8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:19.686862 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405c10d-1f61-4f0c-b20f-aa666f6c81f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:19.688011 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405c10d-1f61-4f0c-b20f-aa666f6c81f8 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:19.689715 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 19/39] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:37:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:19.690566 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405c10d-1f61-4f0c-b20f-aa666f6c81f8 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:19.698371 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:19.784714 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8efec0-ad33-4046-916d-7165516140e9 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:19.813437 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405c10d-1f61-4f0c-b20f-aa666f6c81f8 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:19.823483 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603c068c-8c7a-4f96-8316-54c57c86a3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:19.824169 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c068c-8c7a-4f96-8316-54c57c86a3c8 None admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:19.824455 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c068c-8c7a-4f96-8316-54c57c86a3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:19.824960 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c068c-8c7a-4f96-8316-54c57c86a3c8 None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:19.825452 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603c068c-8c7a-4f96-8316-54c57c86a3c8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:19.826851 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603c068c-8c7a-4f96-8316-54c57c86a3c8 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:19.845633 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 21/40] 10.222.0.20 () {60 vars in 1131 bytes} [Tue Jul 26 12:37:19 2022] POST /identity/v3/roles => generated 223 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:37:19.846717 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbd0fc-d8fa-447e-ae79-d438134168c8 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:19.847011 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbd0fc-d8fa-447e-ae79-d438134168c8 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:19.847298 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbd0fc-d8fa-447e-ae79-d438134168c8 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:19.848695 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 22/41] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:37:19.849444 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dbd0fc-d8fa-447e-ae79-d438134168c8 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:19.859367 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85416bb9-7102-4535-af40-494fe799f61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:19.860307 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85416bb9-7102-4535-af40-494fe799f61c None admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:19.860846 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85416bb9-7102-4535-af40-494fe799f61c None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:19.861133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85416bb9-7102-4535-af40-494fe799f61c None admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:19.861852 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85416bb9-7102-4535-af40-494fe799f61c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:19.863775 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85416bb9-7102-4535-af40-494fe799f61c None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:19.958841 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:19.979340 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dbd0fc-d8fa-447e-ae79-d438134168c8 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:20.015308 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 20/42] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Jul 26 12:37:19 2022] POST /identity/v3/projects => generated 301 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:37:20.016803 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce4fb94-7646-4b5f-904b-ae758e127466 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:20.017191 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce4fb94-7646-4b5f-904b-ae758e127466 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:20.017568 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce4fb94-7646-4b5f-904b-ae758e127466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:20.018885 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5270b47-20f1-4acc-8f09-de99f89a4de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:20.019679 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5270b47-20f1-4acc-8f09-de99f89a4de1 None admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:20.020058 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5270b47-20f1-4acc-8f09-de99f89a4de1 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:20.020411 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5270b47-20f1-4acc-8f09-de99f89a4de1 None admin] PATH_INFO: `/v3/domains/default/users/26e313f5057b4411a71a6bd67e68cb0e/roles/0e78444e43c744668178971af4087ca0` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:20.022579 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5270b47-20f1-4acc-8f09-de99f89a4de1 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=26e313f5057b4411a71a6bd67e68cb0e, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:20.024807 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5270b47-20f1-4acc-8f09-de99f89a4de1 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:20.040323 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-7ce4fb94-7646-4b5f-904b-ae758e127466 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:20.042195 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-7ce4fb94-7646-4b5f-904b-ae758e127466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:20.044678 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 23/43] 10.222.0.20 () {58 vars in 1361 bytes} [Tue Jul 26 12:37:19 2022] PUT /identity/v3/domains/default/users/26e313f5057b4411a71a6bd67e68cb0e/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:37:20.069611 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:20.133000 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 21/44] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:37:20.157428 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5270b47-20f1-4acc-8f09-de99f89a4de1 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:20.322310 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5270b47-20f1-4acc-8f09-de99f89a4de1 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:20.360021 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef5dc9d-dc25-46e1-b825-55ba9db47312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:20.360683 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef5dc9d-dc25-46e1-b825-55ba9db47312 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:20.360965 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef5dc9d-dc25-46e1-b825-55ba9db47312 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:20.361246 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef5dc9d-dc25-46e1-b825-55ba9db47312 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:20.362392 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef5dc9d-dc25-46e1-b825-55ba9db47312 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:20.364138 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef5dc9d-dc25-46e1-b825-55ba9db47312 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:20.366855 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 24/45] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:37:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:20.372467 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce4fb94-7646-4b5f-904b-ae758e127466 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:20.472387 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce4fb94-7646-4b5f-904b-ae758e127466 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:20.505977 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1870f512-8448-4426-b3a4-d6e905b69955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:20.506601 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1870f512-8448-4426-b3a4-d6e905b69955 None admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:20.506887 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1870f512-8448-4426-b3a4-d6e905b69955 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:20.507163 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1870f512-8448-4426-b3a4-d6e905b69955 None admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:20.508503 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1870f512-8448-4426-b3a4-d6e905b69955 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:20.510089 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1870f512-8448-4426-b3a4-d6e905b69955 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:20.537013 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 22/46] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Jul 26 12:37:20 2022] POST /identity/v3/projects => generated 312 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:37:22.087979 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9722a-05c7-4faa-9a2b-46b2c0736107 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:22.088498 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9722a-05c7-4faa-9a2b-46b2c0736107 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:22.088605 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9722a-05c7-4faa-9a2b-46b2c0736107 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:22.089892 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 25/47] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:37:22.095144 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e8886-314e-46f4-8b51-f2e4b0c19115 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:22.095579 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e8886-314e-46f4-8b51-f2e4b0c19115 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:22.095841 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e8886-314e-46f4-8b51-f2e4b0c19115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:22.114977 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-7c7e8886-314e-46f4-8b51-f2e4b0c19115 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:22.117038 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-7c7e8886-314e-46f4-8b51-f2e4b0c19115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:22.121244 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:22.201930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 23/48] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:37:22.224468 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b9722a-05c7-4faa-9a2b-46b2c0736107 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:22.241883 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:22.336104 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b9722a-05c7-4faa-9a2b-46b2c0736107 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:22.369841 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34774086-4e63-48c5-9ed6-9effa12d6a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:22.370407 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34774086-4e63-48c5-9ed6-9effa12d6a9f None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:22.370692 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34774086-4e63-48c5-9ed6-9effa12d6a9f None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:22.370964 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34774086-4e63-48c5-9ed6-9effa12d6a9f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:22.382560 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34774086-4e63-48c5-9ed6-9effa12d6a9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:37:22.389286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-34774086-4e63-48c5-9ed6-9effa12d6a9f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:37:22.389286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:37:22.389286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:37:22.389286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:37:22.389286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:37:22.389286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:37:22.389286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:37:22.389286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:37:22.389286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:37:22.389286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:37:22.389286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:37:22.389286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:37:22.389286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:37:22.389286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:37:22.389286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:37:22.389286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:37:22.390149 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:37:22.390149 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:37:22.390149 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:37:22.390149 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:37:22.390149 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:37:22.390149 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:37:22.390149 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:37:22.390149 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:37:22.390149 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:37:22.390149 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:37:22.390149 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:37:22.390149 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:37:22.390149 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:37:22.390149 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:37:22.390149 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:37:22.390149 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:37:22.390982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:37:22.390982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:37:22.390982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:37:22.390982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:37:22.390982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:37:22.390982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:37:22.390982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:37:22.390982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:37:22.390982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:37:22.390982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:37:22.390982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:37:22.390982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:37:22.390982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 26/49] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 26 12:37:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:37:22.396272 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e8886-314e-46f4-8b51-f2e4b0c19115 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:22.499736 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e8886-314e-46f4-8b51-f2e4b0c19115 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:22.533778 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd772787-a171-4e4f-9be4-345382028a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:22.534397 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd772787-a171-4e4f-9be4-345382028a5a None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:22.534681 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd772787-a171-4e4f-9be4-345382028a5a None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:22.534962 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd772787-a171-4e4f-9be4-345382028a5a None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:22.535714 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd772787-a171-4e4f-9be4-345382028a5a None admin] RBAC: Adding query filter params (name=admin) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:22.536023 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd772787-a171-4e4f-9be4-345382028a5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:22.537427 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd772787-a171-4e4f-9be4-345382028a5a None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:22.542759 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:22.546818 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 24/50] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:37:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:22.552364 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34774086-4e63-48c5-9ed6-9effa12d6a9f None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:22.553671 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277490c9-8959-4209-84bf-ef925325a5bf None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:22.553963 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277490c9-8959-4209-84bf-ef925325a5bf None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:22.554244 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277490c9-8959-4209-84bf-ef925325a5bf None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:22.555557 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 25/51] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 12:37:22.560588 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd90d4-bb7a-404d-865d-9921a5ca52cb None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:22.560877 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd90d4-bb7a-404d-865d-9921a5ca52cb None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:22.561158 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd90d4-bb7a-404d-865d-9921a5ca52cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:22.579606 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-a8dd90d4-bb7a-404d-865d-9921a5ca52cb None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:22.580823 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-a8dd90d4-bb7a-404d-865d-9921a5ca52cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:22.627822 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:22.650259 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 26/52] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 12:37:22.653628 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34774086-4e63-48c5-9ed6-9effa12d6a9f None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:22.672882 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd90d4-bb7a-404d-865d-9921a5ca52cb None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:22.687468 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45c700b-1c33-4001-bff1-d7aa8fbebbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:22.688148 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c700b-1c33-4001-bff1-d7aa8fbebbd2 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:22.688436 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c700b-1c33-4001-bff1-d7aa8fbebbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:22.688716 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c700b-1c33-4001-bff1-d7aa8fbebbd2 None admin] PATH_INFO: `/v3/users/26e313f5057b4411a71a6bd67e68cb0e` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:22.689902 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45c700b-1c33-4001-bff1-d7aa8fbebbd2 None admin] RBAC: Authorizing `identity:get_user(user_id=26e313f5057b4411a71a6bd67e68cb0e)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:22.691376 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45c700b-1c33-4001-bff1-d7aa8fbebbd2 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:22.693216 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 27/53] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:37:22 2022] GET /identity/v3/users/26e313f5057b4411a71a6bd67e68cb0e => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:37:22.698835 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45c700b-1c33-4001-bff1-d7aa8fbebbd2 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:22.770787 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd90d4-bb7a-404d-865d-9921a5ca52cb None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:22.798772 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:22.799338 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45c700b-1c33-4001-bff1-d7aa8fbebbd2 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:22.804939 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d86b5-6069-4a2d-aeb3-7e296f16ceae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:22.805916 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d86b5-6069-4a2d-aeb3-7e296f16ceae None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:22.806208 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d86b5-6069-4a2d-aeb3-7e296f16ceae None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:22.806488 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d86b5-6069-4a2d-aeb3-7e296f16ceae None admin] PATH_INFO: `/v3/domains/default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:22.807659 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1d86b5-6069-4a2d-aeb3-7e296f16ceae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:22.809142 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1d86b5-6069-4a2d-aeb3-7e296f16ceae None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:22.810739 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 27/54] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:37:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:37:22.816232 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d86b5-6069-4a2d-aeb3-7e296f16ceae None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:22.833961 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9519a429-5322-40fe-b230-337f5fb61dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:22.834602 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519a429-5322-40fe-b230-337f5fb61dba None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:22.834889 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519a429-5322-40fe-b230-337f5fb61dba None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:22.835167 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519a429-5322-40fe-b230-337f5fb61dba None admin] PATH_INFO: `/v3/domains/default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:22.836828 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9519a429-5322-40fe-b230-337f5fb61dba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:22.838287 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9519a429-5322-40fe-b230-337f5fb61dba None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:22.839910 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 28/55] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:37:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:37:22.845560 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9519a429-5322-40fe-b230-337f5fb61dba None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:22.859785 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:22.921191 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d86b5-6069-4a2d-aeb3-7e296f16ceae None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:22.948683 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9519a429-5322-40fe-b230-337f5fb61dba None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:22.955072 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d024414-c26a-4174-9d35-46f1c8231469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:22.955725 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d024414-c26a-4174-9d35-46f1c8231469 None admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:22.956051 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d024414-c26a-4174-9d35-46f1c8231469 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:22.956335 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d024414-c26a-4174-9d35-46f1c8231469 None admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:22.957018 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d024414-c26a-4174-9d35-46f1c8231469 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:22.958491 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d024414-c26a-4174-9d35-46f1c8231469 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:22.983251 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86394f1-9a8b-4ad8-bb31-f3820e30a8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:22.983902 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86394f1-9a8b-4ad8-bb31-f3820e30a8b8 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:22.984207 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86394f1-9a8b-4ad8-bb31-f3820e30a8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:22.984470 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86394f1-9a8b-4ad8-bb31-f3820e30a8b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:22.984600 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 28/56] 10.222.0.20 () {60 vars in 1140 bytes} [Tue Jul 26 12:37:22 2022] POST /identity/v3/projects => generated 298 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:37:22.985264 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86394f1-9a8b-4ad8-bb31-f3820e30a8b8 None admin] RBAC: Adding query filter params (role.id=0e78444e43c744668178971af4087ca0, scope.domain.id=default, user.id=26e313f5057b4411a71a6bd67e68cb0e) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:22.985578 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86394f1-9a8b-4ad8-bb31-f3820e30a8b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:22.987119 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86394f1-9a8b-4ad8-bb31-f3820e30a8b8 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:22.997524 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 29/57] 10.222.0.20 () {56 vars in 1322 bytes} [Tue Jul 26 12:37:22 2022] GET /identity/v3/role_assignments?user.id=26e313f5057b4411a71a6bd67e68cb0e&scope.domain.id=default&role.id=0e78444e43c744668178971af4087ca0 => generated 530 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:37:25.013803 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733930f-43bc-4b8e-a595-cb92fb14a820 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:25.014326 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733930f-43bc-4b8e-a595-cb92fb14a820 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:25.014407 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733930f-43bc-4b8e-a595-cb92fb14a820 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:25.015733 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 29/58] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:37:25.020979 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f0485-cbe4-4449-9305-f9729b673161 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:25.021385 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f0485-cbe4-4449-9305-f9729b673161 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:25.021550 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f0485-cbe4-4449-9305-f9729b673161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:25.040274 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-061f0485-cbe4-4449-9305-f9729b673161 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:25.041475 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-061f0485-cbe4-4449-9305-f9729b673161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:25.106102 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 30/59] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:37:25.130610 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0733930f-43bc-4b8e-a595-cb92fb14a820 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:25.182693 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:25.234046 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0733930f-43bc-4b8e-a595-cb92fb14a820 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:25.266274 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04024430-ea6f-4532-9946-11bbb515d0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:25.266832 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04024430-ea6f-4532-9946-11bbb515d0ba None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:25.267123 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04024430-ea6f-4532-9946-11bbb515d0ba None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:25.267400 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04024430-ea6f-4532-9946-11bbb515d0ba None admin] PATH_INFO: `/v3/domains/default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:25.268574 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04024430-ea6f-4532-9946-11bbb515d0ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:25.270039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04024430-ea6f-4532-9946-11bbb515d0ba None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:25.271627 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 30/60] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:37:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:25.277753 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f0485-cbe4-4449-9305-f9729b673161 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:25.280843 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:25.373556 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f0485-cbe4-4449-9305-f9729b673161 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:25.405997 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dbad73-177e-445b-a0e0-baf671525478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:25.406603 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbad73-177e-445b-a0e0-baf671525478 None admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:25.406886 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbad73-177e-445b-a0e0-baf671525478 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:25.407161 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbad73-177e-445b-a0e0-baf671525478 None admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:25.407879 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dbad73-177e-445b-a0e0-baf671525478 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:25.409338 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dbad73-177e-445b-a0e0-baf671525478 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:25.435216 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 31/61] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Jul 26 12:37:25 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:37:28.535042 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b600617-2490-4553-ab5c-8898c1ec54e7 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:28.536100 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b600617-2490-4553-ab5c-8898c1ec54e7 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:28.536597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b600617-2490-4553-ab5c-8898c1ec54e7 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:28.538390 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 31/62] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:37:28.544700 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c6d583-e63a-4902-bd5a-2b7d89f4d542 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:28.545658 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c6d583-e63a-4902-bd5a-2b7d89f4d542 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:28.546155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c6d583-e63a-4902-bd5a-2b7d89f4d542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:28.577018 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-61c6d583-e63a-4902-bd5a-2b7d89f4d542 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:28.578560 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-61c6d583-e63a-4902-bd5a-2b7d89f4d542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:28.662535 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:28.675528 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 32/63] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:37:28.711079 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b600617-2490-4553-ab5c-8898c1ec54e7 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:28.731562 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4610254-4cd6-44f4-aecb-5ddfce630124 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:28.731876 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4610254-4cd6-44f4-aecb-5ddfce630124 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:28.732100 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4610254-4cd6-44f4-aecb-5ddfce630124 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:28.733719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 33/64] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:37:28.739145 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ff729-b063-49ba-a66f-6cc98e40b52d None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:28.739436 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ff729-b063-49ba-a66f-6cc98e40b52d None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:28.739775 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ff729-b063-49ba-a66f-6cc98e40b52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:28.770858 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-2d6ff729-b063-49ba-a66f-6cc98e40b52d None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:28.772888 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-2d6ff729-b063-49ba-a66f-6cc98e40b52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:28.865504 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b600617-2490-4553-ab5c-8898c1ec54e7 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:28.871160 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 34/65] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 12:37:28.872853 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bae8b4-8dbb-4a15-9e48-395f1f86304b None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:28.873397 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bae8b4-8dbb-4a15-9e48-395f1f86304b None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:28.873872 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bae8b4-8dbb-4a15-9e48-395f1f86304b None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:28.880228 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 35/66] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:37:28.883459 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45f9cf-c87c-4ed8-b0f5-0869794ad6eb None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:28.883988 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45f9cf-c87c-4ed8-b0f5-0869794ad6eb None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:28.884478 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45f9cf-c87c-4ed8-b0f5-0869794ad6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:28.909851 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-ec45f9cf-c87c-4ed8-b0f5-0869794ad6eb None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:28.911854 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-ec45f9cf-c87c-4ed8-b0f5-0869794ad6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:28.921126 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d7bba1-fab4-47e1-96c2-eabff45056b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:28.921781 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7bba1-fab4-47e1-96c2-eabff45056b2 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:28.922068 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7bba1-fab4-47e1-96c2-eabff45056b2 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:28.922351 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7bba1-fab4-47e1-96c2-eabff45056b2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:28.923241 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:28.938570 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d7bba1-fab4-47e1-96c2-eabff45056b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:37:28.945515 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-c4d7bba1-fab4-47e1-96c2-eabff45056b2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 12:37:28.945515 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:37:28.945515 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:37:28.945515 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:37:28.945515 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:37:28.945515 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:37:28.945515 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:37:28.945515 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:37:28.945515 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:37:28.945515 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:37:28.945515 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:37:28.945515 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:37:28.945515 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:37:28.945515 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:37:28.945515 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:37:28.945515 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:37:28.946445 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:37:28.946445 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:37:28.946445 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:37:28.946445 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:37:28.946445 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:37:28.946445 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:37:28.946445 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:37:28.946445 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:37:28.946445 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:37:28.946445 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:37:28.946445 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:37:28.946445 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:37:28.946445 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:37:28.946445 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:37:28.946445 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:37:28.946445 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:37:28.947205 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:37:28.947205 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:37:28.947205 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:37:28.947205 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:37:28.947205 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:37:28.947205 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:37:28.947205 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:37:28.947205 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:37:28.947205 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:37:28.947205 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:37:28.947205 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 12:37:28.947205 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:37:28.951716 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 32/67] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:37:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:37:28.952771 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d7bba1-fab4-47e1-96c2-eabff45056b2 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:29.003693 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 36/68] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:37:29.003693 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec45f9cf-c87c-4ed8-b0f5-0869794ad6eb None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:29.060019 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:29.089076 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d7bba1-fab4-47e1-96c2-eabff45056b2 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:29.115856 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec45f9cf-c87c-4ed8-b0f5-0869794ad6eb None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:29.127613 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b2344-5741-44c2-b5bb-8a670859d67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:29.128452 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b2344-5741-44c2-b5bb-8a670859d67b None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:29.128735 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b2344-5741-44c2-b5bb-8a670859d67b None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:29.129013 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b2344-5741-44c2-b5bb-8a670859d67b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:29.140137 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4b2344-5741-44c2-b5bb-8a670859d67b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:37:29.145493 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-5a4b2344-5741-44c2-b5bb-8a670859d67b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:37:29.145493 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:37:29.145493 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:37:29.145493 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:37:29.145493 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:37:29.145493 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:37:29.145493 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:37:29.145493 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:37:29.145493 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:37:29.145493 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:37:29.145493 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:37:29.145493 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:37:29.145493 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:37:29.145493 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:37:29.145493 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:37:29.145493 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:37:29.146322 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:37:29.146322 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:37:29.146322 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:37:29.146322 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:37:29.146322 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:37:29.146322 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:37:29.146322 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:37:29.146322 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:37:29.146322 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:37:29.146322 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:37:29.146322 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:37:29.146322 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:37:29.146322 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:37:29.146322 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:37:29.146322 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:37:29.146322 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:37:29.147067 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:37:29.147067 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:37:29.147067 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:37:29.147067 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:37:29.147067 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:37:29.147067 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:37:29.147067 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:37:29.147067 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:37:29.147067 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:37:29.147067 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:37:29.147067 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:37:29.147067 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:37:29.147067 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 33/69] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 26 12:37:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:37:29.148805 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b2344-5741-44c2-b5bb-8a670859d67b None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:29.158731 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499c09b-a587-4290-8637-e7c5d81ab221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:29.160685 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499c09b-a587-4290-8637-e7c5d81ab221 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:29.160685 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499c09b-a587-4290-8637-e7c5d81ab221 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:29.160685 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499c09b-a587-4290-8637-e7c5d81ab221 None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:29.161024 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5499c09b-a587-4290-8637-e7c5d81ab221 None admin] RBAC: Adding query filter params (name=member) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:29.161327 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5499c09b-a587-4290-8637-e7c5d81ab221 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:29.163415 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5499c09b-a587-4290-8637-e7c5d81ab221 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:29.173194 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 37/70] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 26 12:37:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:29.174411 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e8f9df-e9c4-4b1f-b378-fe673199b58a None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:29.174697 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e8f9df-e9c4-4b1f-b378-fe673199b58a None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:29.174970 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e8f9df-e9c4-4b1f-b378-fe673199b58a None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:29.176200 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 38/71] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:37:29.181616 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e8f9df-e9c4-4b1f-b378-fe673199b58a None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:29.200172 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:29.267935 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b2344-5741-44c2-b5bb-8a670859d67b None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:29.297795 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e8f9df-e9c4-4b1f-b378-fe673199b58a None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:29.301517 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cb6b3e-abeb-4ccb-9305-8efd856879bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:29.302161 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb6b3e-abeb-4ccb-9305-8efd856879bd None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:29.302448 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb6b3e-abeb-4ccb-9305-8efd856879bd None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:29.302725 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb6b3e-abeb-4ccb-9305-8efd856879bd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:29.310999 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:29.313625 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cb6b3e-abeb-4ccb-9305-8efd856879bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:37:29.319066 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-c6cb6b3e-abeb-4ccb-9305-8efd856879bd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 12:37:29.319066 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:37:29.319066 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:37:29.319066 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:37:29.319066 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:37:29.319066 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:37:29.319066 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:37:29.319066 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:37:29.319066 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:37:29.319066 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:37:29.319066 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:37:29.319066 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:37:29.319066 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:37:29.319066 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:37:29.319066 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:37:29.319066 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:37:29.320711 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:37:29.320711 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:37:29.320711 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:37:29.320711 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:37:29.320711 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:37:29.320711 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:37:29.320711 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:37:29.320711 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:37:29.320711 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:37:29.320711 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:37:29.320711 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:37:29.320711 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:37:29.320711 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:37:29.320711 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:37:29.320711 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:37:29.320711 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:37:29.321684 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:37:29.321684 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:37:29.321684 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:37:29.321684 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:37:29.321684 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:37:29.321684 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:37:29.321684 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:37:29.321684 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:37:29.321684 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:37:29.321684 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:37:29.321684 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 12:37:29.321684 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:37:29.321684 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 34/72] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 26 12:37:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:37:29.322650 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a0bb2-1898-40ae-a956-bc2b7bf64ece None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:29.322650 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a0bb2-1898-40ae-a956-bc2b7bf64ece None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:29.322650 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a0bb2-1898-40ae-a956-bc2b7bf64ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:29.337256 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a87352-5974-429a-8dad-a240fd0cefa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:29.337863 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a87352-5974-429a-8dad-a240fd0cefa8 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:29.338148 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a87352-5974-429a-8dad-a240fd0cefa8 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:29.338422 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a87352-5974-429a-8dad-a240fd0cefa8 None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:29.339174 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a87352-5974-429a-8dad-a240fd0cefa8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:29.339468 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a87352-5974-429a-8dad-a240fd0cefa8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:29.340274 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-706a0bb2-1898-40ae-a956-bc2b7bf64ece None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:29.340990 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a87352-5974-429a-8dad-a240fd0cefa8 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:29.341480 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-706a0bb2-1898-40ae-a956-bc2b7bf64ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:29.349475 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 39/73] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:37:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:29.350278 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a87352-5974-429a-8dad-a240fd0cefa8 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:29.412350 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 35/74] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:37:29.413543 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9291918-b112-4520-8f74-e4972acf39d8 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:29.414333 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9291918-b112-4520-8f74-e4972acf39d8 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:29.414626 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9291918-b112-4520-8f74-e4972acf39d8 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:29.415931 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 36/75] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:37:29.416627 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9291918-b112-4520-8f74-e4972acf39d8 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:29.444351 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:29.456046 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a87352-5974-429a-8dad-a240fd0cefa8 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:29.490687 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72878645-59ec-41a3-a268-74f4af55019c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:29.491330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72878645-59ec-41a3-a268-74f4af55019c None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:29.491619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72878645-59ec-41a3-a268-74f4af55019c None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:29.491926 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72878645-59ec-41a3-a268-74f4af55019c None admin] PATH_INFO: `/v3/users/8fecfbe4ea8e4bd29589fe594487625c` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:29.506772 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72878645-59ec-41a3-a268-74f4af55019c None admin] RBAC: Authorizing `identity:get_user(user_id=8fecfbe4ea8e4bd29589fe594487625c)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:29.508834 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72878645-59ec-41a3-a268-74f4af55019c None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:29.510494 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 40/76] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:37:29 2022] GET /identity/v3/users/8fecfbe4ea8e4bd29589fe594487625c => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:29.511259 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72878645-59ec-41a3-a268-74f4af55019c None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:29.519532 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9291918-b112-4520-8f74-e4972acf39d8 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:29.545259 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:29.555599 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b56350b-0507-4cd5-a41f-439ff7d29f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:29.556147 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56350b-0507-4cd5-a41f-439ff7d29f52 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:29.556433 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56350b-0507-4cd5-a41f-439ff7d29f52 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:29.556712 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56350b-0507-4cd5-a41f-439ff7d29f52 None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:29.557438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b56350b-0507-4cd5-a41f-439ff7d29f52 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:29.557747 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b56350b-0507-4cd5-a41f-439ff7d29f52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:29.559152 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b56350b-0507-4cd5-a41f-439ff7d29f52 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:29.567254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 37/77] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:37:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:29.568460 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a0e23-979a-4e21-b2f8-7607080c6d97 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:29.568751 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a0e23-979a-4e21-b2f8-7607080c6d97 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:29.569030 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a0e23-979a-4e21-b2f8-7607080c6d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:29.586667 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-0e2a0e23-979a-4e21-b2f8-7607080c6d97 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:29.587914 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-0e2a0e23-979a-4e21-b2f8-7607080c6d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:29.617938 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72878645-59ec-41a3-a268-74f4af55019c None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:29.652976 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23b92e8-87c0-4d26-80c8-19207d607919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:29.653616 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b92e8-87c0-4d26-80c8-19207d607919 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:29.653895 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b92e8-87c0-4d26-80c8-19207d607919 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:29.654166 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b92e8-87c0-4d26-80c8-19207d607919 None admin] PATH_INFO: `/v3/users/26e313f5057b4411a71a6bd67e68cb0e` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:29.655315 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23b92e8-87c0-4d26-80c8-19207d607919 None admin] RBAC: Authorizing `identity:get_user(user_id=26e313f5057b4411a71a6bd67e68cb0e)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:29.655608 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 38/78] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:37:29.656854 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23b92e8-87c0-4d26-80c8-19207d607919 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:29.662998 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 41/79] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:37:29 2022] GET /identity/v3/users/26e313f5057b4411a71a6bd67e68cb0e => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:29.662998 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23b92e8-87c0-4d26-80c8-19207d607919 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:29.663171 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a0e23-979a-4e21-b2f8-7607080c6d97 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:29.682636 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:29.760082 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:29.763299 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a0e23-979a-4e21-b2f8-7607080c6d97 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:29.763568 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23b92e8-87c0-4d26-80c8-19207d607919 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:29.798268 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ca7e7-ef4d-4be9-97c4-3ff24e7bb2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:29.798473 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0f1dbd-f478-4656-9b98-9fc9f077473a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:29.798886 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ca7e7-ef4d-4be9-97c4-3ff24e7bb2c2 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:29.799103 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f1dbd-f478-4656-9b98-9fc9f077473a None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:29.799193 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ca7e7-ef4d-4be9-97c4-3ff24e7bb2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:29.799391 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f1dbd-f478-4656-9b98-9fc9f077473a None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:29.799474 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ca7e7-ef4d-4be9-97c4-3ff24e7bb2c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:29.799694 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f1dbd-f478-4656-9b98-9fc9f077473a None admin] PATH_INFO: `/v3/domains/default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:29.800837 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0f1dbd-f478-4656-9b98-9fc9f077473a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:29.802315 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0f1dbd-f478-4656-9b98-9fc9f077473a None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:29.803956 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 39/80] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:37:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:29.804754 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0f1dbd-f478-4656-9b98-9fc9f077473a None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:29.867714 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ca7e7-ef4d-4be9-97c4-3ff24e7bb2c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:29.869191 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ca7e7-ef4d-4be9-97c4-3ff24e7bb2c2 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:29.893992 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:29.908959 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0f1dbd-f478-4656-9b98-9fc9f077473a None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:29.942741 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4375304d-4777-45bf-917f-99164d089c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:29.943299 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375304d-4777-45bf-917f-99164d089c71 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:29.943586 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375304d-4777-45bf-917f-99164d089c71 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:29.943890 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375304d-4777-45bf-917f-99164d089c71 None admin] PATH_INFO: `/v3/users/8fecfbe4ea8e4bd29589fe594487625c` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:29.945055 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4375304d-4777-45bf-917f-99164d089c71 None admin] RBAC: Authorizing `identity:get_user(user_id=8fecfbe4ea8e4bd29589fe594487625c)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:29.946977 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4375304d-4777-45bf-917f-99164d089c71 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:29.948604 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 40/81] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:37:29 2022] GET /identity/v3/users/8fecfbe4ea8e4bd29589fe594487625c => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:29.953788 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4375304d-4777-45bf-917f-99164d089c71 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:29.966685 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:29.975000 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 42/82] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:37:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 12:37:29.975000 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ca7e7-ef4d-4be9-97c4-3ff24e7bb2c2 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:30.057818 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4375304d-4777-45bf-917f-99164d089c71 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:30.069932 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ca7e7-ef4d-4be9-97c4-3ff24e7bb2c2 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:30.091622 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db69b93d-35c9-4668-a31a-6179fdabc1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:30.092270 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db69b93d-35c9-4668-a31a-6179fdabc1e5 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:30.092557 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db69b93d-35c9-4668-a31a-6179fdabc1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:30.092831 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db69b93d-35c9-4668-a31a-6179fdabc1e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:30.104556 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3acee1-e7b0-4e79-bb5b-6061c9f9153d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:30.105201 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3acee1-e7b0-4e79-bb5b-6061c9f9153d None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:30.105484 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3acee1-e7b0-4e79-bb5b-6061c9f9153d None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:30.105763 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3acee1-e7b0-4e79-bb5b-6061c9f9153d None admin] PATH_INFO: `/v3/roles/member` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:30.116849 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3acee1-e7b0-4e79-bb5b-6061c9f9153d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:37:30.118550 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:30.123286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-1d3acee1-e7b0-4e79-bb5b-6061c9f9153d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 12:37:30.123286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:37:30.123286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:37:30.123286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:37:30.123286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:37:30.123286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:37:30.123286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:37:30.123286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:37:30.123286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:37:30.123286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:37:30.123286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:37:30.123286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:37:30.123286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:37:30.123286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:37:30.123286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:37:30.123286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:37:30.124477 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:37:30.124477 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:37:30.124477 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:37:30.124477 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:37:30.124477 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:37:30.124477 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:37:30.124477 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:37:30.124477 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:37:30.124477 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:37:30.124477 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:37:30.124477 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:37:30.124477 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:37:30.124477 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:37:30.124477 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:37:30.124477 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:37:30.124477 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:37:30.125272 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:37:30.125272 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:37:30.125272 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:37:30.125272 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:37:30.125272 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:37:30.125272 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:37:30.125272 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:37:30.125272 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:37:30.125272 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:37:30.125272 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:37:30.125272 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 12:37:30.125272 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:37:30.125272 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 43/83] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:37:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:37:30.125272 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3acee1-e7b0-4e79-bb5b-6061c9f9153d None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:30.167301 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db69b93d-35c9-4668-a31a-6179fdabc1e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:30.168855 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db69b93d-35c9-4668-a31a-6179fdabc1e5 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:30.187554 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:30.228348 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3acee1-e7b0-4e79-bb5b-6061c9f9153d None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:30.262523 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11d2c2-205b-44c3-8b2c-ca980c4b975e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:30.263146 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11d2c2-205b-44c3-8b2c-ca980c4b975e None admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:30.263433 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11d2c2-205b-44c3-8b2c-ca980c4b975e None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:30.263741 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11d2c2-205b-44c3-8b2c-ca980c4b975e None admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:30.264489 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f11d2c2-205b-44c3-8b2c-ca980c4b975e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:30.265950 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f11d2c2-205b-44c3-8b2c-ca980c4b975e None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:30.267484 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 41/84] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:37:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 12:37:30.268429 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db69b93d-35c9-4668-a31a-6179fdabc1e5 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:30.287326 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 44/85] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Jul 26 12:37:30 2022] POST /identity/v3/users => generated 289 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:37:30.288565 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11d2c2-205b-44c3-8b2c-ca980c4b975e None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:30.334856 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:30.386927 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db69b93d-35c9-4668-a31a-6179fdabc1e5 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:30.424676 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4452aba4-64b9-4fd3-bd5e-48144ed0e864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:30.425309 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452aba4-64b9-4fd3-bd5e-48144ed0e864 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:30.425593 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452aba4-64b9-4fd3-bd5e-48144ed0e864 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:30.425867 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452aba4-64b9-4fd3-bd5e-48144ed0e864 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:30.427387 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11d2c2-205b-44c3-8b2c-ca980c4b975e None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:30.445621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:30.472369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03809a74-650c-4653-8c25-588921f5520c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:30.473039 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03809a74-650c-4653-8c25-588921f5520c None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:30.473323 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03809a74-650c-4653-8c25-588921f5520c None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:30.473600 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03809a74-650c-4653-8c25-588921f5520c None admin] PATH_INFO: `/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:30.475281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03809a74-650c-4653-8c25-588921f5520c None admin] RBAC: Authorizing `identity:get_project(project_id=f5aedd2d5ee342aa80dfd96a062df0d9)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:30.477230 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03809a74-650c-4653-8c25-588921f5520c None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:30.479750 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 45/86] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:37:30 2022] GET /identity/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:30.480137 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03809a74-650c-4653-8c25-588921f5520c None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:30.517365 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4452aba4-64b9-4fd3-bd5e-48144ed0e864 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:30.519035 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4452aba4-64b9-4fd3-bd5e-48144ed0e864 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:30.588041 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:30.607253 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03809a74-650c-4653-8c25-588921f5520c None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:30.646964 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 42/87] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:37:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 12:37:30.647991 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4452aba4-64b9-4fd3-bd5e-48144ed0e864 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:30.656561 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee0700-211f-4187-8ac2-f029abb03ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:30.657521 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee0700-211f-4187-8ac2-f029abb03ddd None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:30.658026 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee0700-211f-4187-8ac2-f029abb03ddd None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:30.658509 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee0700-211f-4187-8ac2-f029abb03ddd None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:30.659440 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ee0700-211f-4187-8ac2-f029abb03ddd None admin] RBAC: Adding query filter params (name=member) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:30.659983 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ee0700-211f-4187-8ac2-f029abb03ddd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:30.661649 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ee0700-211f-4187-8ac2-f029abb03ddd None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:30.671008 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 46/88] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 26 12:37:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:30.672335 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee0700-211f-4187-8ac2-f029abb03ddd None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:30.732186 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:30.774625 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4452aba4-64b9-4fd3-bd5e-48144ed0e864 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:30.794561 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee0700-211f-4187-8ac2-f029abb03ddd None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:30.813565 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f81a596-b079-4d63-b215-f1610ab21534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:30.814563 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81a596-b079-4d63-b215-f1610ab21534 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:30.815049 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81a596-b079-4d63-b215-f1610ab21534 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:30.815525 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81a596-b079-4d63-b215-f1610ab21534 None admin] PATH_INFO: `/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:30.816970 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f81a596-b079-4d63-b215-f1610ab21534 None admin] RBAC: Authorizing `identity:get_project(project_id=f5aedd2d5ee342aa80dfd96a062df0d9)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:30.818741 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f81a596-b079-4d63-b215-f1610ab21534 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:30.821234 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 43/89] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:37:30 2022] GET /identity/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:30.822219 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f81a596-b079-4d63-b215-f1610ab21534 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:30.838057 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d7ae3c-955b-42ed-a218-8e09c2eb946c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:30.838925 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7ae3c-955b-42ed-a218-8e09c2eb946c None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:30.839211 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7ae3c-955b-42ed-a218-8e09c2eb946c None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:30.839488 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7ae3c-955b-42ed-a218-8e09c2eb946c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:30.840885 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d7ae3c-955b-42ed-a218-8e09c2eb946c None admin] RBAC: Adding query filter params (role.id=01c9179cca2747a6b95b0decfca8b7ae, scope.project.id=f5aedd2d5ee342aa80dfd96a062df0d9, user.id=8fecfbe4ea8e4bd29589fe594487625c) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:30.841203 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d7ae3c-955b-42ed-a218-8e09c2eb946c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:30.842932 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d7ae3c-955b-42ed-a218-8e09c2eb946c None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:30.850607 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 47/90] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:37:30 2022] GET /identity/v3/role_assignments?user.id=8fecfbe4ea8e4bd29589fe594487625c&scope.project.id=f5aedd2d5ee342aa80dfd96a062df0d9&role.id=01c9179cca2747a6b95b0decfca8b7ae => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:30.851955 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d7ae3c-955b-42ed-a218-8e09c2eb946c None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:30.860483 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:30.978742 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f81a596-b079-4d63-b215-f1610ab21534 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:31.022394 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08849e0c-4c52-4438-86ed-04198913aebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:31.022996 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08849e0c-4c52-4438-86ed-04198913aebe None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:31.023865 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08849e0c-4c52-4438-86ed-04198913aebe None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:31.025292 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08849e0c-4c52-4438-86ed-04198913aebe None admin] PATH_INFO: `/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:31.027496 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08849e0c-4c52-4438-86ed-04198913aebe None admin] RBAC: Authorizing `identity:get_project(project_id=f5aedd2d5ee342aa80dfd96a062df0d9)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:31.029005 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08849e0c-4c52-4438-86ed-04198913aebe None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:31.031039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 44/91] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:37:30 2022] GET /identity/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:31.031952 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08849e0c-4c52-4438-86ed-04198913aebe None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:31.052055 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d7ae3c-955b-42ed-a218-8e09c2eb946c None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:31.064850 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:31.120183 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898c808f-07a1-479e-8015-fa1bb56520d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:31.121131 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c808f-07a1-479e-8015-fa1bb56520d8 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:31.121559 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c808f-07a1-479e-8015-fa1bb56520d8 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:31.121978 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c808f-07a1-479e-8015-fa1bb56520d8 None admin] PATH_INFO: `/v3/users/8fecfbe4ea8e4bd29589fe594487625c` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:31.124170 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898c808f-07a1-479e-8015-fa1bb56520d8 None admin] RBAC: Authorizing `identity:get_user(user_id=8fecfbe4ea8e4bd29589fe594487625c)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:31.126055 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898c808f-07a1-479e-8015-fa1bb56520d8 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:31.129190 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 48/92] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:37:30 2022] GET /identity/v3/users/8fecfbe4ea8e4bd29589fe594487625c => generated 275 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:31.130964 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898c808f-07a1-479e-8015-fa1bb56520d8 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:31.181393 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:31.195777 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08849e0c-4c52-4438-86ed-04198913aebe None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:31.265772 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bd06f6-9177-46cc-ba14-82226baa16f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:31.266402 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bd06f6-9177-46cc-ba14-82226baa16f4 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:31.266876 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bd06f6-9177-46cc-ba14-82226baa16f4 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:31.267160 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bd06f6-9177-46cc-ba14-82226baa16f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:31.268825 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bd06f6-9177-46cc-ba14-82226baa16f4 None admin] RBAC: Adding query filter params (role.id=0e78444e43c744668178971af4087ca0, scope.project.id=f5aedd2d5ee342aa80dfd96a062df0d9, user.id=26e313f5057b4411a71a6bd67e68cb0e) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:31.269144 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bd06f6-9177-46cc-ba14-82226baa16f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:31.271335 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bd06f6-9177-46cc-ba14-82226baa16f4 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:31.278455 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 45/93] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:37:31 2022] GET /identity/v3/role_assignments?user.id=26e313f5057b4411a71a6bd67e68cb0e&scope.project.id=f5aedd2d5ee342aa80dfd96a062df0d9&role.id=0e78444e43c744668178971af4087ca0 => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:31.279349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bd06f6-9177-46cc-ba14-82226baa16f4 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:31.393938 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898c808f-07a1-479e-8015-fa1bb56520d8 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:31.458625 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d0403b-1887-4ebd-9249-48ba36d42685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:31.459248 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d0403b-1887-4ebd-9249-48ba36d42685 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:31.459527 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d0403b-1887-4ebd-9249-48ba36d42685 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:31.459824 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d0403b-1887-4ebd-9249-48ba36d42685 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:31.460619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d0403b-1887-4ebd-9249-48ba36d42685 None admin] RBAC: Adding query filter params (role.id=6589779ddd834447a1ed1921e1c834f6, scope.project.id=f5aedd2d5ee342aa80dfd96a062df0d9, user.id=8fecfbe4ea8e4bd29589fe594487625c) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:31.461027 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d0403b-1887-4ebd-9249-48ba36d42685 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:31.462476 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d0403b-1887-4ebd-9249-48ba36d42685 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:31.474017 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 49/94] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:37:31 2022] GET /identity/v3/role_assignments?user.id=8fecfbe4ea8e4bd29589fe594487625c&scope.project.id=f5aedd2d5ee342aa80dfd96a062df0d9&role.id=6589779ddd834447a1ed1921e1c834f6 => generated 259 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:31.476138 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bd06f6-9177-46cc-ba14-82226baa16f4 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:31.537306 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223c9906-f0a1-4360-a192-81d67af969be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:31.537877 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c9906-f0a1-4360-a192-81d67af969be None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:31.538162 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c9906-f0a1-4360-a192-81d67af969be None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:31.538436 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c9906-f0a1-4360-a192-81d67af969be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:31.571688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:31.644911 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223c9906-f0a1-4360-a192-81d67af969be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:31.646427 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223c9906-f0a1-4360-a192-81d67af969be None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:31.783541 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 46/95] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:37:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 12:37:31.789749 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d0403b-1887-4ebd-9249-48ba36d42685 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:31.855836 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:31.931762 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d0403b-1887-4ebd-9249-48ba36d42685 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:31.979674 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec39fd1-554e-407c-aa70-189f53395085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:31.980350 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec39fd1-554e-407c-aa70-189f53395085 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:31.980637 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec39fd1-554e-407c-aa70-189f53395085 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:31.980914 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec39fd1-554e-407c-aa70-189f53395085 None admin] PATH_INFO: `/v3/projects/cf365e8707b941b0b695d30a70dce1af` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:31.988393 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec39fd1-554e-407c-aa70-189f53395085 None admin] RBAC: Authorizing `identity:get_project(project_id=cf365e8707b941b0b695d30a70dce1af)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:31.990150 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec39fd1-554e-407c-aa70-189f53395085 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:31.992613 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 50/96] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:37:31 2022] GET /identity/v3/projects/cf365e8707b941b0b695d30a70dce1af => generated 312 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:31.998947 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223c9906-f0a1-4360-a192-81d67af969be None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:32.103869 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:32.167775 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223c9906-f0a1-4360-a192-81d67af969be None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:32.212101 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18325ab9-74b2-427d-ba02-213ba4e3d802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:32.213079 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18325ab9-74b2-427d-ba02-213ba4e3d802 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:32.213771 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18325ab9-74b2-427d-ba02-213ba4e3d802 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:32.214248 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18325ab9-74b2-427d-ba02-213ba4e3d802 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:32.215302 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18325ab9-74b2-427d-ba02-213ba4e3d802 None admin] RBAC: Adding query filter params (role.id=01c9179cca2747a6b95b0decfca8b7ae, scope.project.id=cf365e8707b941b0b695d30a70dce1af, user.id=8fecfbe4ea8e4bd29589fe594487625c) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:32.219955 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18325ab9-74b2-427d-ba02-213ba4e3d802 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:32.221734 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18325ab9-74b2-427d-ba02-213ba4e3d802 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:32.229252 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 47/97] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:37:31 2022] GET /identity/v3/role_assignments?user.id=8fecfbe4ea8e4bd29589fe594487625c&scope.project.id=cf365e8707b941b0b695d30a70dce1af&role.id=01c9179cca2747a6b95b0decfca8b7ae => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:33.100711 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c071e1-213d-4cb0-9fae-ac105d521f01 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:33.101758 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c071e1-213d-4cb0-9fae-ac105d521f01 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:33.101758 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c071e1-213d-4cb0-9fae-ac105d521f01 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:33.103232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 51/98] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:37:33.108606 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98967200-52a7-493d-a76f-04d67ce2bad9 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:33.109035 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98967200-52a7-493d-a76f-04d67ce2bad9 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:33.109199 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98967200-52a7-493d-a76f-04d67ce2bad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:33.128457 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-98967200-52a7-493d-a76f-04d67ce2bad9 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:33.129646 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-98967200-52a7-493d-a76f-04d67ce2bad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:33.187625 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6905fc7-66eb-449d-9f6d-4195b2374441 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:33.187979 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6905fc7-66eb-449d-9f6d-4195b2374441 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:33.188221 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6905fc7-66eb-449d-9f6d-4195b2374441 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:33.189650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 52/99] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 12:37:33.206370 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeb3434-11d8-4d1e-aa0f-6598a3682565 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:33.206678 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeb3434-11d8-4d1e-aa0f-6598a3682565 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:33.206961 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeb3434-11d8-4d1e-aa0f-6598a3682565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:33.240694 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-6eeb3434-11d8-4d1e-aa0f-6598a3682565 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:33.241931 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-6eeb3434-11d8-4d1e-aa0f-6598a3682565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:33.259412 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 48/100] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:37:33.329568 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98967200-52a7-493d-a76f-04d67ce2bad9 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:33.381434 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 53/101] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:37:33.424712 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:33.447539 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeb3434-11d8-4d1e-aa0f-6598a3682565 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:33.508516 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98967200-52a7-493d-a76f-04d67ce2bad9 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:33.549727 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:33.572437 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b02e31-fccc-411d-9541-88531c76a103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:33.573093 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b02e31-fccc-411d-9541-88531c76a103 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:33.573379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b02e31-fccc-411d-9541-88531c76a103 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:33.573649 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b02e31-fccc-411d-9541-88531c76a103 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:33.574804 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b02e31-fccc-411d-9541-88531c76a103 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:33.576300 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b02e31-fccc-411d-9541-88531c76a103 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:33.578232 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 49/102] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:37:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:37:33.585292 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b02e31-fccc-411d-9541-88531c76a103 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:33.679932 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeb3434-11d8-4d1e-aa0f-6598a3682565 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:33.765346 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961cb51a-d36e-4fd1-9312-793209a7501d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:33.766012 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961cb51a-d36e-4fd1-9312-793209a7501d None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:33.766292 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961cb51a-d36e-4fd1-9312-793209a7501d None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:33.766563 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961cb51a-d36e-4fd1-9312-793209a7501d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:33.798447 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961cb51a-d36e-4fd1-9312-793209a7501d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:37:33.798807 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b02e31-fccc-411d-9541-88531c76a103 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:33.803547 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-961cb51a-d36e-4fd1-9312-793209a7501d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 12:37:33.803547 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:37:33.803547 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:37:33.803547 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:37:33.803547 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:37:33.803547 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:37:33.803547 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:37:33.803547 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:37:33.803547 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:37:33.803547 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:37:33.803547 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:37:33.803547 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:37:33.803547 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:37:33.803547 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:37:33.803547 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:37:33.803547 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:37:33.804493 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:37:33.804493 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:37:33.804493 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:37:33.804493 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:37:33.804493 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:37:33.804493 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:37:33.804493 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:37:33.804493 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:37:33.804493 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:37:33.804493 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:37:33.804493 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:37:33.804493 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:37:33.804493 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:37:33.804493 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:37:33.804493 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:37:33.804493 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:37:33.805244 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:37:33.805244 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:37:33.805244 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:37:33.805244 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:37:33.805244 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:37:33.805244 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:37:33.805244 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:37:33.805244 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:37:33.805244 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:37:33.805244 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:37:33.805244 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 12:37:33.805244 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:37:33.805244 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 54/103] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:37:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:37:33.811118 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c1bd83-ebd9-4c46-a9ee-e64cfc61b8aa None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:33.811457 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c1bd83-ebd9-4c46-a9ee-e64cfc61b8aa None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:33.811759 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c1bd83-ebd9-4c46-a9ee-e64cfc61b8aa None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:33.813628 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 55/104] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:37:33.819493 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c1bd83-ebd9-4c46-a9ee-e64cfc61b8aa None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:33.840072 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:33.852479 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc92e631-9866-47ce-b57c-8fe3e7f9f6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:33.853112 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc92e631-9866-47ce-b57c-8fe3e7f9f6df None admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:33.853401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc92e631-9866-47ce-b57c-8fe3e7f9f6df None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:33.853743 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc92e631-9866-47ce-b57c-8fe3e7f9f6df None admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:33.854461 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc92e631-9866-47ce-b57c-8fe3e7f9f6df None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:33.856015 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc92e631-9866-47ce-b57c-8fe3e7f9f6df None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:33.891279 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 50/105] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Jul 26 12:37:33 2022] POST /identity/v3/projects => generated 302 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:37:33.893129 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0e092-155e-4bf4-8705-cdc51c45269d None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:33.893465 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0e092-155e-4bf4-8705-cdc51c45269d None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:33.893809 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0e092-155e-4bf4-8705-cdc51c45269d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:33.915993 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-5bf0e092-155e-4bf4-8705-cdc51c45269d None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:33.917204 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-5bf0e092-155e-4bf4-8705-cdc51c45269d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:33.999275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 51/106] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:37:34.023400 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0e092-155e-4bf4-8705-cdc51c45269d None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:34.060532 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:34.071158 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c1bd83-ebd9-4c46-a9ee-e64cfc61b8aa None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:34.126929 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e3e5e2-78f9-4960-9575-c7fee6507f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:34.128383 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3e5e2-78f9-4960-9575-c7fee6507f32 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:34.129509 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3e5e2-78f9-4960-9575-c7fee6507f32 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:34.129981 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3e5e2-78f9-4960-9575-c7fee6507f32 None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:34.130873 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e3e5e2-78f9-4960-9575-c7fee6507f32 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:34.131309 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e3e5e2-78f9-4960-9575-c7fee6507f32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:34.133041 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e3e5e2-78f9-4960-9575-c7fee6507f32 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:34.144975 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 56/107] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 26 12:37:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:34.152666 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e3e5e2-78f9-4960-9575-c7fee6507f32 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:34.225076 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:34.237665 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0e092-155e-4bf4-8705-cdc51c45269d None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:34.289329 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e3e5e2-78f9-4960-9575-c7fee6507f32 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:34.328839 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea89077e-4b91-4bc4-9d54-8e16f014d8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:34.329850 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89077e-4b91-4bc4-9d54-8e16f014d8c8 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:34.330301 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89077e-4b91-4bc4-9d54-8e16f014d8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:34.330778 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89077e-4b91-4bc4-9d54-8e16f014d8c8 None admin] PATH_INFO: `/v3/users/b6ca251cd8f24a4fa7f5aa93cd0c29ac` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:34.341146 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a684da0-eda0-4541-85e7-7a70ab1999df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:34.341973 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a684da0-eda0-4541-85e7-7a70ab1999df None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:34.342353 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a684da0-eda0-4541-85e7-7a70ab1999df None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:34.343247 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a684da0-eda0-4541-85e7-7a70ab1999df None admin] PATH_INFO: `/v3/roles/member` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:34.351002 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea89077e-4b91-4bc4-9d54-8e16f014d8c8 None admin] RBAC: Authorizing `identity:get_user(user_id=b6ca251cd8f24a4fa7f5aa93cd0c29ac)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:34.353739 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea89077e-4b91-4bc4-9d54-8e16f014d8c8 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:34.355663 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 57/108] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:37:34 2022] GET /identity/v3/users/b6ca251cd8f24a4fa7f5aa93cd0c29ac => generated 289 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:34.361703 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea89077e-4b91-4bc4-9d54-8e16f014d8c8 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:34.370726 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a684da0-eda0-4541-85e7-7a70ab1999df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:37:34.382401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-6a684da0-eda0-4541-85e7-7a70ab1999df None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 12:37:34.382401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:37:34.382401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:37:34.382401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:37:34.382401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:37:34.382401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:37:34.382401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:37:34.382401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:37:34.382401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:37:34.382401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:37:34.382401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:37:34.382401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:37:34.382401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:37:34.382401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:37:34.382401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:37:34.382401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:37:34.383417 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:37:34.383417 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:37:34.383417 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:37:34.383417 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:37:34.383417 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:37:34.383417 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:37:34.383417 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:37:34.383417 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:37:34.383417 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:37:34.383417 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:37:34.383417 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:37:34.383417 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:37:34.383417 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:37:34.383417 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:37:34.383417 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:37:34.383417 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:37:34.388843 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:37:34.388843 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:37:34.388843 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:37:34.388843 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:37:34.388843 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:37:34.388843 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:37:34.388843 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:37:34.388843 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:37:34.388843 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:37:34.388843 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:37:34.388843 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 12:37:34.388843 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:37:34.388843 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 52/109] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:37:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:37:34.395540 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a684da0-eda0-4541-85e7-7a70ab1999df None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:34.396104 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:34.502153 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a684da0-eda0-4541-85e7-7a70ab1999df None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:34.509802 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea89077e-4b91-4bc4-9d54-8e16f014d8c8 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:34.544406 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567309e9-8aa8-48fc-bc27-64169c1fbe7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:34.545111 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567309e9-8aa8-48fc-bc27-64169c1fbe7e None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:34.545399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567309e9-8aa8-48fc-bc27-64169c1fbe7e None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:34.545677 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567309e9-8aa8-48fc-bc27-64169c1fbe7e None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:34.546756 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567309e9-8aa8-48fc-bc27-64169c1fbe7e None admin] RBAC: Adding query filter params (name=member) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:34.547129 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567309e9-8aa8-48fc-bc27-64169c1fbe7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:34.548850 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567309e9-8aa8-48fc-bc27-64169c1fbe7e None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:34.556126 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e2d702-ce62-4139-be57-a6f134367628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:34.558531 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2d702-ce62-4139-be57-a6f134367628 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:34.559285 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2d702-ce62-4139-be57-a6f134367628 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:34.559989 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2d702-ce62-4139-be57-a6f134367628 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:34.565800 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 53/110] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 26 12:37:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:34.565800 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c05ef-d1b0-4a0d-90ae-a45356b13f14 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:34.565800 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c05ef-d1b0-4a0d-90ae-a45356b13f14 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:34.565800 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c05ef-d1b0-4a0d-90ae-a45356b13f14 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:34.565800 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 54/111] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:37:34.568780 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377441fc-9acd-437f-be5f-f556e487b87b None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:34.569213 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377441fc-9acd-437f-be5f-f556e487b87b None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:34.569617 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377441fc-9acd-437f-be5f-f556e487b87b None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:34.573442 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 55/112] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:37:34.574431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377441fc-9acd-437f-be5f-f556e487b87b None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:34.608010 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:34.648086 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e2d702-ce62-4139-be57-a6f134367628 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:34.650356 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e2d702-ce62-4139-be57-a6f134367628 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:34.673269 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:34.692509 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377441fc-9acd-437f-be5f-f556e487b87b None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:34.730244 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a220a-dd44-46a0-af19-6f1239db97fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:34.731233 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a220a-dd44-46a0-af19-6f1239db97fd None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:34.731792 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a220a-dd44-46a0-af19-6f1239db97fd None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:34.732273 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a220a-dd44-46a0-af19-6f1239db97fd None admin] PATH_INFO: `/v3/users/8fecfbe4ea8e4bd29589fe594487625c` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:34.733764 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a220a-dd44-46a0-af19-6f1239db97fd None admin] RBAC: Authorizing `identity:get_user(user_id=8fecfbe4ea8e4bd29589fe594487625c)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:34.735536 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a220a-dd44-46a0-af19-6f1239db97fd None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:34.737512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 56/113] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:37:34 2022] GET /identity/v3/users/8fecfbe4ea8e4bd29589fe594487625c => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:34.739425 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b5738d-14cd-416d-a231-224ad1989687 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:34.739954 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b5738d-14cd-416d-a231-224ad1989687 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:34.740448 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b5738d-14cd-416d-a231-224ad1989687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:34.759176 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 58/114] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:37:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 12:37:34.760623 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5cf6b1-7212-4611-b9fb-493dd57a6563 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:34.760910 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5cf6b1-7212-4611-b9fb-493dd57a6563 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:34.761187 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5cf6b1-7212-4611-b9fb-493dd57a6563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:34.761467 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-75b5738d-14cd-416d-a231-224ad1989687 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:34.762868 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-75b5738d-14cd-416d-a231-224ad1989687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:34.779614 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-fc5cf6b1-7212-4611-b9fb-493dd57a6563 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:34.780846 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-fc5cf6b1-7212-4611-b9fb-493dd57a6563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:34.845228 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 57/115] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:37:34.846574 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b5738d-14cd-416d-a231-224ad1989687 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:34.880471 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:34.881208 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 59/116] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:37:34.882128 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5cf6b1-7212-4611-b9fb-493dd57a6563 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:34.939646 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:34.963165 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b5738d-14cd-416d-a231-224ad1989687 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:34.992529 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5cf6b1-7212-4611-b9fb-493dd57a6563 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:35.001965 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e80144-d7cb-4cab-a077-0270349bc935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:35.003095 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e80144-d7cb-4cab-a077-0270349bc935 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:35.003597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e80144-d7cb-4cab-a077-0270349bc935 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:35.004119 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e80144-d7cb-4cab-a077-0270349bc935 None admin] PATH_INFO: `/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:35.005732 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e80144-d7cb-4cab-a077-0270349bc935 None admin] RBAC: Authorizing `identity:get_project(project_id=f5aedd2d5ee342aa80dfd96a062df0d9)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:35.008254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e80144-d7cb-4cab-a077-0270349bc935 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:35.010641 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 58/117] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:37:34 2022] GET /identity/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:35.011868 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e80144-d7cb-4cab-a077-0270349bc935 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:35.029618 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e9e97-6991-4cb2-b317-c4d6cabbeed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:35.030304 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e9e97-6991-4cb2-b317-c4d6cabbeed9 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:35.030542 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e9e97-6991-4cb2-b317-c4d6cabbeed9 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:35.030817 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e9e97-6991-4cb2-b317-c4d6cabbeed9 None admin] PATH_INFO: `/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:35.032008 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9e9e97-6991-4cb2-b317-c4d6cabbeed9 None admin] RBAC: Authorizing `identity:get_project(project_id=f5aedd2d5ee342aa80dfd96a062df0d9)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:35.033615 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9e9e97-6991-4cb2-b317-c4d6cabbeed9 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:35.035302 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 60/118] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:37:34 2022] GET /identity/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:35.035917 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e9e97-6991-4cb2-b317-c4d6cabbeed9 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:35.112077 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:35.125935 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e80144-d7cb-4cab-a077-0270349bc935 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:35.138056 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e9e97-6991-4cb2-b317-c4d6cabbeed9 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:35.156171 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:35.168120 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce09adc-9e23-4b25-8a58-5a77e4f7c3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:35.169235 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce09adc-9e23-4b25-8a58-5a77e4f7c3f7 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:35.170103 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce09adc-9e23-4b25-8a58-5a77e4f7c3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:35.170603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce09adc-9e23-4b25-8a58-5a77e4f7c3f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:35.173727 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1752da-79bb-4f0b-b961-aeb06fc410c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:35.174372 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1752da-79bb-4f0b-b961-aeb06fc410c8 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:35.174653 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1752da-79bb-4f0b-b961-aeb06fc410c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:35.174929 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1752da-79bb-4f0b-b961-aeb06fc410c8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:35.185508 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce09adc-9e23-4b25-8a58-5a77e4f7c3f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:37:35.186771 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1752da-79bb-4f0b-b961-aeb06fc410c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:37:35.190526 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-4ce09adc-9e23-4b25-8a58-5a77e4f7c3f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:37:35.190526 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:37:35.190526 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:37:35.190526 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:37:35.190526 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:37:35.190526 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:37:35.190526 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:37:35.190526 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:37:35.190526 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:37:35.190526 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:37:35.190526 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:37:35.190526 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:37:35.190526 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:37:35.190526 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:37:35.190526 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:37:35.190526 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:37:35.191383 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:37:35.191383 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:37:35.191383 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:37:35.191383 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:37:35.191383 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:37:35.191383 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:37:35.191383 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:37:35.191383 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:37:35.191383 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:37:35.191383 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:37:35.191383 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:37:35.191383 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:37:35.191383 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:37:35.191383 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:37:35.191383 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:37:35.191383 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:37:35.192207 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:37:35.192207 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:37:35.192207 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:37:35.192207 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:37:35.192207 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:37:35.192207 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:37:35.192207 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:37:35.192207 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:37:35.192207 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:37:35.192207 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:37:35.192207 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:37:35.192207 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:37:35.192818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-ba1752da-79bb-4f0b-b961-aeb06fc410c8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 12:37:35.192818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:37:35.192818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:37:35.192818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:37:35.192818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:37:35.192818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:37:35.192818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:37:35.192818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:37:35.192818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:37:35.192818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:37:35.192818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:37:35.192818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:37:35.192818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:37:35.192818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:37:35.192818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:37:35.192818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:37:35.193618 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:37:35.193618 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:37:35.193618 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:37:35.193618 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:37:35.193618 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:37:35.193618 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:37:35.193618 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:37:35.193618 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:37:35.193618 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:37:35.193618 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:37:35.193618 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:37:35.193618 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:37:35.193618 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:37:35.193618 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:37:35.193618 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:37:35.193618 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:37:35.194372 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:37:35.194372 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:37:35.194372 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:37:35.194372 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:37:35.194372 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:37:35.194372 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:37:35.194372 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:37:35.194372 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:37:35.194372 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:37:35.194372 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:37:35.194372 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 12:37:35.194372 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:37:35.194968 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 59/119] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 26 12:37:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:37:35.194968 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce09adc-9e23-4b25-8a58-5a77e4f7c3f7 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:35.198039 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 61/120] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 26 12:37:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:37:35.199159 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1752da-79bb-4f0b-b961-aeb06fc410c8 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:35.304067 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce09adc-9e23-4b25-8a58-5a77e4f7c3f7 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:35.305868 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1752da-79bb-4f0b-b961-aeb06fc410c8 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:35.339733 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c98e9-1438-4705-a349-92bdcbe85151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:35.340403 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c98e9-1438-4705-a349-92bdcbe85151 None admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:35.340683 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c98e9-1438-4705-a349-92bdcbe85151 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:35.340955 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c98e9-1438-4705-a349-92bdcbe85151 None admin] PATH_INFO: `/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9/users/8fecfbe4ea8e4bd29589fe594487625c/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:35.341208 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d33b14-bab0-4a9f-96ee-9163c3a1c6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:35.341828 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d33b14-bab0-4a9f-96ee-9163c3a1c6dc None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:35.342116 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d33b14-bab0-4a9f-96ee-9163c3a1c6dc None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:35.342394 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d33b14-bab0-4a9f-96ee-9163c3a1c6dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:35.343008 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368c98e9-1438-4705-a349-92bdcbe85151 None admin] RBAC: Authorizing `identity:create_grant(project_id=f5aedd2d5ee342aa80dfd96a062df0d9, user_id=8fecfbe4ea8e4bd29589fe594487625c, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:35.343176 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d33b14-bab0-4a9f-96ee-9163c3a1c6dc None admin] RBAC: Adding query filter params (role.id=db16c43424694218852e80f51fb1ee70, scope.project.id=f5aedd2d5ee342aa80dfd96a062df0d9, user.id=b6ca251cd8f24a4fa7f5aa93cd0c29ac) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:35.343588 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d33b14-bab0-4a9f-96ee-9163c3a1c6dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:35.344521 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368c98e9-1438-4705-a349-92bdcbe85151 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:35.345074 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d33b14-bab0-4a9f-96ee-9163c3a1c6dc None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:35.358773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 62/121] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:37:35 2022] GET /identity/v3/role_assignments?user.id=b6ca251cd8f24a4fa7f5aa93cd0c29ac&scope.project.id=f5aedd2d5ee342aa80dfd96a062df0d9&role.id=db16c43424694218852e80f51fb1ee70 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:35.358773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8355c-ec18-40a4-94f7-e75e7605793e None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:35.358773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8355c-ec18-40a4-94f7-e75e7605793e None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:35.358773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8355c-ec18-40a4-94f7-e75e7605793e None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:35.358773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 63/122] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:37:35.358773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c8355c-ec18-40a4-94f7-e75e7605793e None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:35.372411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 60/123] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 26 12:37:35 2022] PUT /identity/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9/users/8fecfbe4ea8e4bd29589fe594487625c/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:37:35.373034 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c98e9-1438-4705-a349-92bdcbe85151 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:35.386746 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:35.431893 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:35.590555 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c98e9-1438-4705-a349-92bdcbe85151 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:35.602010 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c8355c-ec18-40a4-94f7-e75e7605793e None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:35.632667 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2195f27-4f07-4156-baff-a99af415a73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:35.633708 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2195f27-4f07-4156-baff-a99af415a73b None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:35.634461 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2195f27-4f07-4156-baff-a99af415a73b None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:35.635223 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2195f27-4f07-4156-baff-a99af415a73b None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:35.636163 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2195f27-4f07-4156-baff-a99af415a73b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:35.636783 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2195f27-4f07-4156-baff-a99af415a73b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:35.639068 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2195f27-4f07-4156-baff-a99af415a73b None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:35.645191 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db032f9-7637-4f8b-9dcd-dd72beef7fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:35.645901 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db032f9-7637-4f8b-9dcd-dd72beef7fe0 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:35.646277 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db032f9-7637-4f8b-9dcd-dd72beef7fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:35.646658 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db032f9-7637-4f8b-9dcd-dd72beef7fe0 None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:35.647515 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db032f9-7637-4f8b-9dcd-dd72beef7fe0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:35.647918 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db032f9-7637-4f8b-9dcd-dd72beef7fe0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:35.649967 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db032f9-7637-4f8b-9dcd-dd72beef7fe0 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:35.652073 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 61/124] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:37:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:35.653865 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c62d3-5f8e-4ed3-a25f-e9f6cc94dfde None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:35.654330 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c62d3-5f8e-4ed3-a25f-e9f6cc94dfde None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:35.654811 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c62d3-5f8e-4ed3-a25f-e9f6cc94dfde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:35.666310 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 64/125] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:37:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:35.667038 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db032f9-7637-4f8b-9dcd-dd72beef7fe0 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:35.700690 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-c08c62d3-5f8e-4ed3-a25f-e9f6cc94dfde None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:35.702000 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-c08c62d3-5f8e-4ed3-a25f-e9f6cc94dfde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:35.833749 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db032f9-7637-4f8b-9dcd-dd72beef7fe0 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:35.834418 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:35.839962 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:35.841222 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 62/126] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:37:35.841877 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08c62d3-5f8e-4ed3-a25f-e9f6cc94dfde None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:35.910281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b485ed-1a69-419a-84c6-ecc6153f6f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:35.910861 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b485ed-1a69-419a-84c6-ecc6153f6f74 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:35.911148 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b485ed-1a69-419a-84c6-ecc6153f6f74 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:35.911427 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b485ed-1a69-419a-84c6-ecc6153f6f74 None admin] PATH_INFO: `/v3/users/8fecfbe4ea8e4bd29589fe594487625c` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:35.914860 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b485ed-1a69-419a-84c6-ecc6153f6f74 None admin] RBAC: Authorizing `identity:get_user(user_id=8fecfbe4ea8e4bd29589fe594487625c)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:35.916380 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b485ed-1a69-419a-84c6-ecc6153f6f74 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:35.923792 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 65/127] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:37:35 2022] GET /identity/v3/users/8fecfbe4ea8e4bd29589fe594487625c => generated 275 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:35.924700 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b485ed-1a69-419a-84c6-ecc6153f6f74 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:36.041616 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08c62d3-5f8e-4ed3-a25f-e9f6cc94dfde None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:36.077472 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b485ed-1a69-419a-84c6-ecc6153f6f74 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:36.119019 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba957d5-eb60-4e1f-8b10-c3cd4db0a912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:36.119687 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba957d5-eb60-4e1f-8b10-c3cd4db0a912 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:36.119980 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba957d5-eb60-4e1f-8b10-c3cd4db0a912 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:36.120263 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba957d5-eb60-4e1f-8b10-c3cd4db0a912 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:36.135347 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d303dd-680f-4e02-8658-3a90a226f4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:36.136029 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d303dd-680f-4e02-8658-3a90a226f4d3 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:36.136324 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d303dd-680f-4e02-8658-3a90a226f4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:36.136606 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d303dd-680f-4e02-8658-3a90a226f4d3 None admin] PATH_INFO: `/v3/users/26e313f5057b4411a71a6bd67e68cb0e` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:36.138741 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d303dd-680f-4e02-8658-3a90a226f4d3 None admin] RBAC: Authorizing `identity:get_user(user_id=26e313f5057b4411a71a6bd67e68cb0e)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:36.140242 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d303dd-680f-4e02-8658-3a90a226f4d3 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:36.141888 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 63/128] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:37:35 2022] GET /identity/v3/users/26e313f5057b4411a71a6bd67e68cb0e => generated 247 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:36.144207 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba957d5-eb60-4e1f-8b10-c3cd4db0a912 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:37:36.144654 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d303dd-680f-4e02-8658-3a90a226f4d3 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:36.150262 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-7ba957d5-eb60-4e1f-8b10-c3cd4db0a912 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 12:37:36.150262 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:37:36.150262 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:37:36.150262 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:37:36.150262 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:37:36.150262 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:37:36.150262 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:37:36.150262 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:37:36.150262 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:37:36.150262 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:37:36.150262 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:37:36.150262 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:37:36.150262 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:37:36.150262 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:37:36.150262 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:37:36.150262 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:37:36.151115 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:37:36.151115 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:37:36.151115 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:37:36.151115 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:37:36.151115 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:37:36.151115 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:37:36.151115 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:37:36.151115 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:37:36.151115 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:37:36.151115 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:37:36.151115 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:37:36.151115 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:37:36.151115 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:37:36.151115 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:37:36.151115 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:37:36.151115 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:37:36.152288 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:37:36.152288 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:37:36.152288 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:37:36.152288 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:37:36.152288 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:37:36.152288 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:37:36.152288 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:37:36.152288 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:37:36.152288 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:37:36.152288 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:37:36.152288 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 12:37:36.152288 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:37:36.152288 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 66/129] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:37:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:37:36.152288 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba957d5-eb60-4e1f-8b10-c3cd4db0a912 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:36.237426 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:36.274214 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:36.314127 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d303dd-680f-4e02-8658-3a90a226f4d3 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:36.388529 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4140826f-3687-4d90-858a-d53250129faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:36.389583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba957d5-eb60-4e1f-8b10-c3cd4db0a912 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:36.389991 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4140826f-3687-4d90-858a-d53250129faf None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:36.390475 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4140826f-3687-4d90-858a-d53250129faf None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:36.390978 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4140826f-3687-4d90-858a-d53250129faf None admin] PATH_INFO: `/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:36.396410 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4140826f-3687-4d90-858a-d53250129faf None admin] RBAC: Authorizing `identity:get_project(project_id=f5aedd2d5ee342aa80dfd96a062df0d9)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:36.398713 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4140826f-3687-4d90-858a-d53250129faf None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:36.400692 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 64/130] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:37:36 2022] GET /identity/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9 => generated 298 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:36.401795 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4140826f-3687-4d90-858a-d53250129faf None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:36.435598 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7658f1-1bff-4c43-b3b9-7e23190959b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:36.436615 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7658f1-1bff-4c43-b3b9-7e23190959b7 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:36.437995 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7658f1-1bff-4c43-b3b9-7e23190959b7 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:36.439852 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7658f1-1bff-4c43-b3b9-7e23190959b7 None admin] PATH_INFO: `/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:36.442518 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7658f1-1bff-4c43-b3b9-7e23190959b7 None admin] RBAC: Authorizing `identity:get_project(project_id=f5aedd2d5ee342aa80dfd96a062df0d9)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:36.444233 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7658f1-1bff-4c43-b3b9-7e23190959b7 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:36.446143 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 67/131] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:37:36 2022] GET /identity/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9 => generated 298 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:36.447439 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7658f1-1bff-4c43-b3b9-7e23190959b7 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:36.557068 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:36.558160 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4140826f-3687-4d90-858a-d53250129faf None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:36.560250 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7658f1-1bff-4c43-b3b9-7e23190959b7 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:36.601674 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f769a-ce01-4b3a-b8fd-2392aa680a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:36.602345 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f769a-ce01-4b3a-b8fd-2392aa680a0d None admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:36.602633 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f769a-ce01-4b3a-b8fd-2392aa680a0d None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:36.602909 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f769a-ce01-4b3a-b8fd-2392aa680a0d None admin] PATH_INFO: `/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9/users/8fecfbe4ea8e4bd29589fe594487625c/roles/6589779ddd834447a1ed1921e1c834f6` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:36.603549 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3954f61b-b0c6-4eb8-b46c-9e0e30e5a658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:36.604375 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954f61b-b0c6-4eb8-b46c-9e0e30e5a658 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:36.604808 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954f61b-b0c6-4eb8-b46c-9e0e30e5a658 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:36.605151 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954f61b-b0c6-4eb8-b46c-9e0e30e5a658 None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:36.605256 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447f769a-ce01-4b3a-b8fd-2392aa680a0d None admin] RBAC: Authorizing `identity:create_grant(project_id=f5aedd2d5ee342aa80dfd96a062df0d9, user_id=8fecfbe4ea8e4bd29589fe594487625c, role_id=6589779ddd834447a1ed1921e1c834f6)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:36.605956 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3954f61b-b0c6-4eb8-b46c-9e0e30e5a658 None admin] RBAC: Adding query filter params (name=member) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:36.606324 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3954f61b-b0c6-4eb8-b46c-9e0e30e5a658 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:36.606621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447f769a-ce01-4b3a-b8fd-2392aa680a0d None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:36.607858 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3954f61b-b0c6-4eb8-b46c-9e0e30e5a658 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:36.610235 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:36.620983 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 65/132] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 26 12:37:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:36.622658 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07e872-31fd-48ab-ab40-59d8439654dd None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:36.623013 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07e872-31fd-48ab-ab40-59d8439654dd None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:36.623359 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07e872-31fd-48ab-ab40-59d8439654dd None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:36.624933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 66/133] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:37:36.625809 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b07e872-31fd-48ab-ab40-59d8439654dd None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:36.632106 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 68/134] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 26 12:37:36 2022] PUT /identity/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9/users/8fecfbe4ea8e4bd29589fe594487625c/roles/6589779ddd834447a1ed1921e1c834f6 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:37:36.633101 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f769a-ce01-4b3a-b8fd-2392aa680a0d None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:36.828217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f769a-ce01-4b3a-b8fd-2392aa680a0d None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:36.850750 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b07e872-31fd-48ab-ab40-59d8439654dd None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:36.877960 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8c5d6-cb2a-4d22-9489-18994b8667ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:36.878870 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8c5d6-cb2a-4d22-9489-18994b8667ca None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:36.879157 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8c5d6-cb2a-4d22-9489-18994b8667ca None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:36.879705 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8c5d6-cb2a-4d22-9489-18994b8667ca None admin] PATH_INFO: `/v3/users/8fecfbe4ea8e4bd29589fe594487625c` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:36.881621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f8c5d6-cb2a-4d22-9489-18994b8667ca None admin] RBAC: Authorizing `identity:get_user(user_id=8fecfbe4ea8e4bd29589fe594487625c)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:36.884256 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f8c5d6-cb2a-4d22-9489-18994b8667ca None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:36.886199 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 69/135] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:37:36 2022] GET /identity/v3/users/8fecfbe4ea8e4bd29589fe594487625c => generated 275 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:36.887322 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1607d-1726-4516-be47-5e75ab194241 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:36.887669 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1607d-1726-4516-be47-5e75ab194241 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:36.887959 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1607d-1726-4516-be47-5e75ab194241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:36.889393 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f0ba2-6a23-4776-b61b-a17e018f569a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:36.890288 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f0ba2-6a23-4776-b61b-a17e018f569a None admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:36.890771 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f0ba2-6a23-4776-b61b-a17e018f569a None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:36.891227 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f0ba2-6a23-4776-b61b-a17e018f569a None admin] PATH_INFO: `/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9/users/26e313f5057b4411a71a6bd67e68cb0e/roles/0e78444e43c744668178971af4087ca0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:36.893472 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74f0ba2-6a23-4776-b61b-a17e018f569a None admin] RBAC: Authorizing `identity:create_grant(project_id=f5aedd2d5ee342aa80dfd96a062df0d9, user_id=26e313f5057b4411a71a6bd67e68cb0e, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:36.895142 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74f0ba2-6a23-4776-b61b-a17e018f569a None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:36.917691 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-a2b1607d-1726-4516-be47-5e75ab194241 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:36.919120 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-a2b1607d-1726-4516-be47-5e75ab194241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:36.925352 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 67/136] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 26 12:37:36 2022] PUT /identity/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9/users/26e313f5057b4411a71a6bd67e68cb0e/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:37:36.927191 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f0ba2-6a23-4776-b61b-a17e018f569a None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:36.957603 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:37.006901 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:37.020078 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 70/137] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:37:37.044861 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1607d-1726-4516-be47-5e75ab194241 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:37.064677 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f0ba2-6a23-4776-b61b-a17e018f569a None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:37.105429 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a25eb-6f42-44a7-8f6e-4351abd581b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:37.106432 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a25eb-6f42-44a7-8f6e-4351abd581b6 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:37.107013 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a25eb-6f42-44a7-8f6e-4351abd581b6 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:37.107489 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a25eb-6f42-44a7-8f6e-4351abd581b6 None admin] PATH_INFO: `/v3/projects/cf365e8707b941b0b695d30a70dce1af` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:37.108915 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0a25eb-6f42-44a7-8f6e-4351abd581b6 None admin] RBAC: Authorizing `identity:get_project(project_id=cf365e8707b941b0b695d30a70dce1af)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:37.110594 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0a25eb-6f42-44a7-8f6e-4351abd581b6 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:37.112576 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 68/138] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:37:36 2022] GET /identity/v3/projects/cf365e8707b941b0b695d30a70dce1af => generated 312 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:37.131813 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a25eb-6f42-44a7-8f6e-4351abd581b6 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:37.172718 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1607d-1726-4516-be47-5e75ab194241 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:37.226601 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71731f2-391e-407d-9807-1e233591ff65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:37.227263 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71731f2-391e-407d-9807-1e233591ff65 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:37.227548 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71731f2-391e-407d-9807-1e233591ff65 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:37.227848 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71731f2-391e-407d-9807-1e233591ff65 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:37.230552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71731f2-391e-407d-9807-1e233591ff65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:37.232204 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71731f2-391e-407d-9807-1e233591ff65 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:37.234021 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 71/139] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:37:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:37:37.241106 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71731f2-391e-407d-9807-1e233591ff65 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:37.315127 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:37.361332 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a25eb-6f42-44a7-8f6e-4351abd581b6 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:37.399984 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71731f2-391e-407d-9807-1e233591ff65 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:37.404015 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:37.415590 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b70b466-a29a-49a6-b2fd-79c6d65ef164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:37.416308 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b70b466-a29a-49a6-b2fd-79c6d65ef164 None admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:37.416599 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b70b466-a29a-49a6-b2fd-79c6d65ef164 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:37.416887 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b70b466-a29a-49a6-b2fd-79c6d65ef164 None admin] PATH_INFO: `/v3/projects/cf365e8707b941b0b695d30a70dce1af/users/8fecfbe4ea8e4bd29589fe594487625c/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:37.419551 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b70b466-a29a-49a6-b2fd-79c6d65ef164 None admin] RBAC: Authorizing `identity:create_grant(project_id=cf365e8707b941b0b695d30a70dce1af, user_id=8fecfbe4ea8e4bd29589fe594487625c, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:37.425600 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b70b466-a29a-49a6-b2fd-79c6d65ef164 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:37.451619 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 69/140] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 26 12:37:37 2022] PUT /identity/v3/projects/cf365e8707b941b0b695d30a70dce1af/users/8fecfbe4ea8e4bd29589fe594487625c/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:37:37.456485 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8487204b-87ba-49e3-84c4-6bff267a2efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:37.457136 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8487204b-87ba-49e3-84c4-6bff267a2efc None admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:37.457422 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8487204b-87ba-49e3-84c4-6bff267a2efc None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:37.457701 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8487204b-87ba-49e3-84c4-6bff267a2efc None admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:37.458434 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8487204b-87ba-49e3-84c4-6bff267a2efc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:37.459989 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8487204b-87ba-49e3-84c4-6bff267a2efc None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:37.490391 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 72/141] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Jul 26 12:37:37 2022] POST /identity/v3/users => generated 283 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:37:37.969428 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef722da-3626-426a-98d9-07e35fdfae4f None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:37.969808 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef722da-3626-426a-98d9-07e35fdfae4f None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:37.970009 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef722da-3626-426a-98d9-07e35fdfae4f None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:37.971492 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 70/142] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:37:37.976985 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01fcf1-41ff-43ef-920c-eb7ebda9cf29 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:37.977509 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01fcf1-41ff-43ef-920c-eb7ebda9cf29 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:37.977976 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01fcf1-41ff-43ef-920c-eb7ebda9cf29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:37.997450 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-3d01fcf1-41ff-43ef-920c-eb7ebda9cf29 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:37.998941 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-3d01fcf1-41ff-43ef-920c-eb7ebda9cf29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:38.048862 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4baf01-0632-419b-bafd-b6520d0e041b None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:38.049465 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4baf01-0632-419b-bafd-b6520d0e041b None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:38.050005 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4baf01-0632-419b-bafd-b6520d0e041b None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:38.051584 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 71/143] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:37:38.057323 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80afbba-14cd-4c63-b4b4-3a89d76fcb56 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:38.057863 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80afbba-14cd-4c63-b4b4-3a89d76fcb56 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:38.058345 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80afbba-14cd-4c63-b4b4-3a89d76fcb56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:38.087757 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-f80afbba-14cd-4c63-b4b4-3a89d76fcb56 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:38.090548 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-f80afbba-14cd-4c63-b4b4-3a89d76fcb56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:38.107737 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 73/144] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:37:38.162703 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01fcf1-41ff-43ef-920c-eb7ebda9cf29 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:38.188459 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:38.230184 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 72/145] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 12:37:38.273176 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80afbba-14cd-4c63-b4b4-3a89d76fcb56 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:38.330013 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01fcf1-41ff-43ef-920c-eb7ebda9cf29 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:38.391273 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b690a66-0d8f-43f9-8b20-21722fd5a990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:38.392312 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b690a66-0d8f-43f9-8b20-21722fd5a990 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:38.392840 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b690a66-0d8f-43f9-8b20-21722fd5a990 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:38.393551 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b690a66-0d8f-43f9-8b20-21722fd5a990 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:38.407142 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b690a66-0d8f-43f9-8b20-21722fd5a990 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:37:38.413985 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-1b690a66-0d8f-43f9-8b20-21722fd5a990 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 12:37:38.413985 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:37:38.413985 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:37:38.413985 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:37:38.413985 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:37:38.413985 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:37:38.413985 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:37:38.413985 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:37:38.413985 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:37:38.413985 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:37:38.413985 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:37:38.413985 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:37:38.413985 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:37:38.413985 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:37:38.413985 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:37:38.413985 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:37:38.414864 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:37:38.414864 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:37:38.414864 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:37:38.414864 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:37:38.414864 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:37:38.414864 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:37:38.414864 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:37:38.414864 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:37:38.414864 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:37:38.414864 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:37:38.414864 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:37:38.414864 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:37:38.414864 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:37:38.414864 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:37:38.414864 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:37:38.414864 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:37:38.415619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:37:38.415619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:37:38.415619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:37:38.415619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:37:38.415619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:37:38.415619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:37:38.415619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:37:38.415619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:37:38.415619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:37:38.415619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:37:38.415619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 12:37:38.415619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:37:38.418102 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 74/146] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:37:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:37:38.426276 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b690a66-0d8f-43f9-8b20-21722fd5a990 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:38.532128 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:38.533563 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80afbba-14cd-4c63-b4b4-3a89d76fcb56 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:38.547758 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b690a66-0d8f-43f9-8b20-21722fd5a990 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:38.578204 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:38.598459 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57769a93-6594-44f7-b128-2e7758baf8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:38.599462 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57769a93-6594-44f7-b128-2e7758baf8b9 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:38.599974 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57769a93-6594-44f7-b128-2e7758baf8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:38.600480 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57769a93-6594-44f7-b128-2e7758baf8b9 None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:38.601906 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57769a93-6594-44f7-b128-2e7758baf8b9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:38.602418 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57769a93-6594-44f7-b128-2e7758baf8b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:38.604077 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57769a93-6594-44f7-b128-2e7758baf8b9 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:38.614400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 75/147] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 26 12:37:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:38.622932 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57769a93-6594-44f7-b128-2e7758baf8b9 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:38.641177 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b47356-e7a5-4cf6-9f94-2f7cc0266795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:38.641814 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b47356-e7a5-4cf6-9f94-2f7cc0266795 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:38.642092 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b47356-e7a5-4cf6-9f94-2f7cc0266795 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:38.642362 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b47356-e7a5-4cf6-9f94-2f7cc0266795 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:38.656547 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b47356-e7a5-4cf6-9f94-2f7cc0266795 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:37:38.664688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-f0b47356-e7a5-4cf6-9f94-2f7cc0266795 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 12:37:38.664688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:37:38.664688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:37:38.664688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:37:38.664688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:37:38.664688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:37:38.664688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:37:38.664688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:37:38.664688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:37:38.664688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:37:38.664688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:37:38.664688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:37:38.664688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:37:38.664688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:37:38.664688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:37:38.664688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:37:38.665621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:37:38.665621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:37:38.665621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:37:38.665621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:37:38.665621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:37:38.665621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:37:38.665621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:37:38.665621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:37:38.665621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:37:38.665621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:37:38.665621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:37:38.665621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:37:38.665621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:37:38.665621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:37:38.665621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:37:38.665621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:37:38.666403 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:37:38.666403 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:37:38.666403 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:37:38.666403 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:37:38.666403 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:37:38.666403 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:37:38.666403 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:37:38.666403 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:37:38.666403 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:37:38.666403 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:37:38.666403 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 12:37:38.666403 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:37:38.666403 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 73/148] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:37:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:37:38.672636 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b47356-e7a5-4cf6-9f94-2f7cc0266795 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:38.752340 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57769a93-6594-44f7-b128-2e7758baf8b9 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:38.796533 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7931e6b-d1a7-4955-8cb5-eead3b0ce6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:38.797096 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7931e6b-d1a7-4955-8cb5-eead3b0ce6f9 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:38.797375 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7931e6b-d1a7-4955-8cb5-eead3b0ce6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:38.797642 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7931e6b-d1a7-4955-8cb5-eead3b0ce6f9 None admin] PATH_INFO: `/v3/users/b6ca251cd8f24a4fa7f5aa93cd0c29ac` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:38.798924 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7931e6b-d1a7-4955-8cb5-eead3b0ce6f9 None admin] RBAC: Authorizing `identity:get_user(user_id=b6ca251cd8f24a4fa7f5aa93cd0c29ac)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:38.800500 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7931e6b-d1a7-4955-8cb5-eead3b0ce6f9 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:38.802287 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 76/149] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:37:38 2022] GET /identity/v3/users/b6ca251cd8f24a4fa7f5aa93cd0c29ac => generated 289 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:38.808281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7931e6b-d1a7-4955-8cb5-eead3b0ce6f9 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:38.812543 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:38.819826 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b47356-e7a5-4cf6-9f94-2f7cc0266795 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:38.867162 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c5eaa8-0a3d-4f02-bc28-29a19cb60c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:38.868385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c5eaa8-0a3d-4f02-bc28-29a19cb60c07 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:38.869340 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c5eaa8-0a3d-4f02-bc28-29a19cb60c07 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:38.869819 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c5eaa8-0a3d-4f02-bc28-29a19cb60c07 None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:38.870975 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c5eaa8-0a3d-4f02-bc28-29a19cb60c07 None admin] RBAC: Adding query filter params (name=member) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:38.871757 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c5eaa8-0a3d-4f02-bc28-29a19cb60c07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:38.873776 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c5eaa8-0a3d-4f02-bc28-29a19cb60c07 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:38.886276 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 74/150] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 26 12:37:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:37:38.895874 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c5eaa8-0a3d-4f02-bc28-29a19cb60c07 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:38.935904 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7931e6b-d1a7-4955-8cb5-eead3b0ce6f9 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:38.947975 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:38.992917 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fd6d6f-cccb-425e-ba27-8c3c5ad7b974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:38.994047 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd6d6f-cccb-425e-ba27-8c3c5ad7b974 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:38.994493 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd6d6f-cccb-425e-ba27-8c3c5ad7b974 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:38.995753 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd6d6f-cccb-425e-ba27-8c3c5ad7b974 None admin] PATH_INFO: `/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:39.001093 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fd6d6f-cccb-425e-ba27-8c3c5ad7b974 None admin] RBAC: Authorizing `identity:get_project(project_id=f5aedd2d5ee342aa80dfd96a062df0d9)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:39.003334 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fd6d6f-cccb-425e-ba27-8c3c5ad7b974 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:39.011034 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 77/151] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:37:38 2022] GET /identity/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:37:39.021430 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fd6d6f-cccb-425e-ba27-8c3c5ad7b974 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:39.064792 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c5eaa8-0a3d-4f02-bc28-29a19cb60c07 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:39.136013 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe4f78-3924-4b46-990f-41f5b2ce3290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:39.137044 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe4f78-3924-4b46-990f-41f5b2ce3290 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:39.137527 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe4f78-3924-4b46-990f-41f5b2ce3290 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:39.138001 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe4f78-3924-4b46-990f-41f5b2ce3290 None admin] PATH_INFO: `/v3/users/8fecfbe4ea8e4bd29589fe594487625c` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:39.139662 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfe4f78-3924-4b46-990f-41f5b2ce3290 None admin] RBAC: Authorizing `identity:get_user(user_id=8fecfbe4ea8e4bd29589fe594487625c)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:39.141326 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfe4f78-3924-4b46-990f-41f5b2ce3290 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:39.144544 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 75/152] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:37:38 2022] GET /identity/v3/users/8fecfbe4ea8e4bd29589fe594487625c => generated 275 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:37:39.150528 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe4f78-3924-4b46-990f-41f5b2ce3290 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:39.173734 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:39.231856 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fd6d6f-cccb-425e-ba27-8c3c5ad7b974 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:39.303020 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06639625-5e54-42f9-9805-06c4ee9cfcdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:39.304121 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06639625-5e54-42f9-9805-06c4ee9cfcdc None admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:39.305695 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06639625-5e54-42f9-9805-06c4ee9cfcdc None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:39.306374 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06639625-5e54-42f9-9805-06c4ee9cfcdc None admin] PATH_INFO: `/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9/users/b6ca251cd8f24a4fa7f5aa93cd0c29ac/roles/db16c43424694218852e80f51fb1ee70` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:39.310029 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06639625-5e54-42f9-9805-06c4ee9cfcdc None admin] RBAC: Authorizing `identity:create_grant(project_id=f5aedd2d5ee342aa80dfd96a062df0d9, user_id=b6ca251cd8f24a4fa7f5aa93cd0c29ac, role_id=db16c43424694218852e80f51fb1ee70)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:39.312707 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06639625-5e54-42f9-9805-06c4ee9cfcdc None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:39.344320 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 78/153] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 26 12:37:39 2022] PUT /identity/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9/users/b6ca251cd8f24a4fa7f5aa93cd0c29ac/roles/db16c43424694218852e80f51fb1ee70 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:37:39.372507 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe4f78-3924-4b46-990f-41f5b2ce3290 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:39.386517 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:39.423226 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224530b-5c68-4794-9fa2-7c869620eac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:39.424241 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224530b-5c68-4794-9fa2-7c869620eac4 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:39.424751 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224530b-5c68-4794-9fa2-7c869620eac4 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:39.425226 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224530b-5c68-4794-9fa2-7c869620eac4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:39.526011 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8224530b-5c68-4794-9fa2-7c869620eac4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:39.527933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8224530b-5c68-4794-9fa2-7c869620eac4 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:39.673625 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 76/154] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:37:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 12:37:39.680088 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06639625-5e54-42f9-9805-06c4ee9cfcdc None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:39.864197 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06639625-5e54-42f9-9805-06c4ee9cfcdc None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:39.905024 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:39.912226 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25f5a0-f1d0-4ff2-8ed0-f6f545f8b1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:39.913175 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25f5a0-f1d0-4ff2-8ed0-f6f545f8b1ab None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:39.914272 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25f5a0-f1d0-4ff2-8ed0-f6f545f8b1ab None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:39.914849 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25f5a0-f1d0-4ff2-8ed0-f6f545f8b1ab None admin] PATH_INFO: `/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:39.916369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d25f5a0-f1d0-4ff2-8ed0-f6f545f8b1ab None admin] RBAC: Authorizing `identity:get_project(project_id=f5aedd2d5ee342aa80dfd96a062df0d9)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:39.918066 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d25f5a0-f1d0-4ff2-8ed0-f6f545f8b1ab None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:39.920303 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 79/155] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:37:39 2022] GET /identity/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9 => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:39.928824 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224530b-5c68-4794-9fa2-7c869620eac4 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:39.942074 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:40.092003 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888aed7d-2459-4aa7-ab3a-03dd6cb92e2d None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:40.092705 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888aed7d-2459-4aa7-ab3a-03dd6cb92e2d None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:40.093190 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888aed7d-2459-4aa7-ab3a-03dd6cb92e2d None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:40.094773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 80/156] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 12:37:40.100053 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dbc14d-5ffd-437e-aac3-d0d9179da90a None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:40.100595 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dbc14d-5ffd-437e-aac3-d0d9179da90a None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:40.101798 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dbc14d-5ffd-437e-aac3-d0d9179da90a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:40.103997 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224530b-5c68-4794-9fa2-7c869620eac4 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:40.138755 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-48dbc14d-5ffd-437e-aac3-d0d9179da90a None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:40.140333 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-48dbc14d-5ffd-437e-aac3-d0d9179da90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:40.175760 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b846b5-913a-47c3-83a4-1172f9172191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:40.176386 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b846b5-913a-47c3-83a4-1172f9172191 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:40.176661 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b846b5-913a-47c3-83a4-1172f9172191 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:40.176930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b846b5-913a-47c3-83a4-1172f9172191 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:40.177730 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b846b5-913a-47c3-83a4-1172f9172191 None admin] RBAC: Adding query filter params (role.id=01c9179cca2747a6b95b0decfca8b7ae, scope.project.id=f5aedd2d5ee342aa80dfd96a062df0d9, user.id=8fecfbe4ea8e4bd29589fe594487625c) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:40.178037 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b846b5-913a-47c3-83a4-1172f9172191 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:40.180136 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b846b5-913a-47c3-83a4-1172f9172191 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:40.197960 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 77/157] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:37:39 2022] GET /identity/v3/role_assignments?user.id=8fecfbe4ea8e4bd29589fe594487625c&scope.project.id=f5aedd2d5ee342aa80dfd96a062df0d9&role.id=01c9179cca2747a6b95b0decfca8b7ae => generated 608 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:40.283359 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 81/158] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:37:40.342019 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b846b5-913a-47c3-83a4-1172f9172191 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:40.443966 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:40.484491 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b846b5-913a-47c3-83a4-1172f9172191 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:40.529881 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c10c71a-0587-49c0-84bd-1a575c57218b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:40.530534 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c10c71a-0587-49c0-84bd-1a575c57218b None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:40.530809 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c10c71a-0587-49c0-84bd-1a575c57218b None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:40.531077 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c10c71a-0587-49c0-84bd-1a575c57218b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:40.548525 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c10c71a-0587-49c0-84bd-1a575c57218b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:37:40.553955 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-7c10c71a-0587-49c0-84bd-1a575c57218b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 12:37:40.553955 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:37:40.553955 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:37:40.553955 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:37:40.553955 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:37:40.553955 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:37:40.553955 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:37:40.553955 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:37:40.553955 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:37:40.553955 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:37:40.553955 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:37:40.553955 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:37:40.553955 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:37:40.553955 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:37:40.553955 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:37:40.553955 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:37:40.554899 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:37:40.554899 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:37:40.554899 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:37:40.554899 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:37:40.554899 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:37:40.554899 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:37:40.554899 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:37:40.554899 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:37:40.554899 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:37:40.554899 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:37:40.554899 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:37:40.554899 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:37:40.554899 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:37:40.554899 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:37:40.554899 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:37:40.554899 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:37:40.555756 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:37:40.555756 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:37:40.555756 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:37:40.555756 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:37:40.555756 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:37:40.555756 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:37:40.555756 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:37:40.555756 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:37:40.555756 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:37:40.555756 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:37:40.555756 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 12:37:40.555756 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:37:40.555756 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 78/159] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 26 12:37:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:37:40.563420 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dbc14d-5ffd-437e-aac3-d0d9179da90a None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:40.743621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dbc14d-5ffd-437e-aac3-d0d9179da90a None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:40.782514 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:40.792800 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a4ab8-dca1-4e2d-bc74-03f64e00dced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:40.793795 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a4ab8-dca1-4e2d-bc74-03f64e00dced None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:40.794277 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a4ab8-dca1-4e2d-bc74-03f64e00dced None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:40.794748 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a4ab8-dca1-4e2d-bc74-03f64e00dced None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:40.795702 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a4ab8-dca1-4e2d-bc74-03f64e00dced None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:40.796209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a4ab8-dca1-4e2d-bc74-03f64e00dced None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:40.797818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a4ab8-dca1-4e2d-bc74-03f64e00dced None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:40.801990 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501dfee-1bc3-4b89-a56d-e261ca64eec4 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:40.801990 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501dfee-1bc3-4b89-a56d-e261ca64eec4 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:40.801990 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501dfee-1bc3-4b89-a56d-e261ca64eec4 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:40.801990 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 79/160] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:37:40.808210 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135bbb25-9e5b-44e8-abbc-d348e4833391 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:40.808491 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135bbb25-9e5b-44e8-abbc-d348e4833391 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:40.808765 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135bbb25-9e5b-44e8-abbc-d348e4833391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:40.815345 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 82/161] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:37:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:40.822654 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a4ab8-dca1-4e2d-bc74-03f64e00dced None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:40.834907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-135bbb25-9e5b-44e8-abbc-d348e4833391 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:40.836632 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-135bbb25-9e5b-44e8-abbc-d348e4833391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:40.973171 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 80/162] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 12:37:41.007673 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a4ab8-dca1-4e2d-bc74-03f64e00dced None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:41.039185 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135bbb25-9e5b-44e8-abbc-d348e4833391 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:41.066836 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe492e-c703-49df-bc63-f9409965b2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:41.067500 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe492e-c703-49df-bc63-f9409965b2c8 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:41.067816 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe492e-c703-49df-bc63-f9409965b2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:41.068138 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe492e-c703-49df-bc63-f9409965b2c8 None admin] PATH_INFO: `/v3/users/8fecfbe4ea8e4bd29589fe594487625c` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:41.070451 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfe492e-c703-49df-bc63-f9409965b2c8 None admin] RBAC: Authorizing `identity:get_user(user_id=8fecfbe4ea8e4bd29589fe594487625c)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:41.071956 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfe492e-c703-49df-bc63-f9409965b2c8 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:41.073641 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 83/163] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:37:40 2022] GET /identity/v3/users/8fecfbe4ea8e4bd29589fe594487625c => generated 275 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:41.079171 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe492e-c703-49df-bc63-f9409965b2c8 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:41.158286 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:41.171446 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:41.225834 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135bbb25-9e5b-44e8-abbc-d348e4833391 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:41.278392 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed935a-478b-472a-a6b4-1e12edff8043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:41.278966 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed935a-478b-472a-a6b4-1e12edff8043 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:41.279245 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed935a-478b-472a-a6b4-1e12edff8043 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:41.279514 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed935a-478b-472a-a6b4-1e12edff8043 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:41.290333 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe492e-c703-49df-bc63-f9409965b2c8 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:41.297370 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed935a-478b-472a-a6b4-1e12edff8043 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:37:41.310471 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-4eed935a-478b-472a-a6b4-1e12edff8043 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:37:41.310471 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:37:41.310471 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:37:41.310471 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:37:41.310471 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:37:41.310471 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:37:41.310471 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:37:41.310471 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:37:41.310471 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:37:41.310471 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:37:41.310471 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:37:41.310471 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:37:41.310471 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:37:41.310471 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:37:41.310471 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:37:41.310471 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:37:41.311340 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:37:41.311340 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:37:41.311340 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:37:41.311340 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:37:41.311340 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:37:41.311340 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:37:41.311340 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:37:41.311340 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:37:41.311340 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:37:41.311340 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:37:41.311340 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:37:41.311340 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:37:41.311340 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:37:41.311340 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:37:41.311340 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:37:41.311340 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:37:41.312145 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:37:41.312145 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:37:41.312145 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:37:41.312145 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:37:41.312145 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:37:41.312145 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:37:41.312145 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:37:41.312145 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:37:41.312145 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:37:41.312145 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:37:41.312145 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:37:41.312145 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:37:41.312145 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 81/164] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 26 12:37:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:37:41.321923 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed935a-478b-472a-a6b4-1e12edff8043 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:41.412376 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0996ecd9-679b-4c1f-8e85-fa9fdc8b535e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:41.413225 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996ecd9-679b-4c1f-8e85-fa9fdc8b535e None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:41.413642 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996ecd9-679b-4c1f-8e85-fa9fdc8b535e None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:41.413923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996ecd9-679b-4c1f-8e85-fa9fdc8b535e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:41.478733 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed935a-478b-472a-a6b4-1e12edff8043 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:41.509697 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:41.537723 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3d076-d9da-4f07-b0ad-b1e17f885212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:41.538913 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3d076-d9da-4f07-b0ad-b1e17f885212 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:41.539676 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3d076-d9da-4f07-b0ad-b1e17f885212 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:41.540163 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3d076-d9da-4f07-b0ad-b1e17f885212 None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:41.541079 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de3d076-d9da-4f07-b0ad-b1e17f885212 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:41.541586 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de3d076-d9da-4f07-b0ad-b1e17f885212 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:41.543170 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de3d076-d9da-4f07-b0ad-b1e17f885212 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:41.559541 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 82/165] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:37:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:37:41.567297 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3d076-d9da-4f07-b0ad-b1e17f885212 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:41.588718 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0996ecd9-679b-4c1f-8e85-fa9fdc8b535e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:41.590200 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0996ecd9-679b-4c1f-8e85-fa9fdc8b535e None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:41.732186 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:41.793152 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3d076-d9da-4f07-b0ad-b1e17f885212 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:41.858719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 84/166] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:37:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 780 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 12:37:41.865682 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0996ecd9-679b-4c1f-8e85-fa9fdc8b535e None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:41.885817 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cef63bf-d626-43a9-8969-e53c4eb887ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:41.886481 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef63bf-d626-43a9-8969-e53c4eb887ca None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:41.886763 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef63bf-d626-43a9-8969-e53c4eb887ca None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:41.887065 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef63bf-d626-43a9-8969-e53c4eb887ca None admin] PATH_INFO: `/v3/users/26e313f5057b4411a71a6bd67e68cb0e` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:41.892542 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cef63bf-d626-43a9-8969-e53c4eb887ca None admin] RBAC: Authorizing `identity:get_user(user_id=26e313f5057b4411a71a6bd67e68cb0e)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:41.894094 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cef63bf-d626-43a9-8969-e53c4eb887ca None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:41.895919 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 83/167] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:37:41 2022] GET /identity/v3/users/26e313f5057b4411a71a6bd67e68cb0e => generated 247 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:37:41.902869 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cef63bf-d626-43a9-8969-e53c4eb887ca None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:42.052190 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:42.064500 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0996ecd9-679b-4c1f-8e85-fa9fdc8b535e None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:42.131854 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cef63bf-d626-43a9-8969-e53c4eb887ca None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:42.133474 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2da390f-896e-40b0-9319-e059b0b11d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:42.134464 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da390f-896e-40b0-9319-e059b0b11d7a None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:42.134971 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da390f-896e-40b0-9319-e059b0b11d7a None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:42.135433 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da390f-896e-40b0-9319-e059b0b11d7a None admin] PATH_INFO: `/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:42.140426 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2da390f-896e-40b0-9319-e059b0b11d7a None admin] RBAC: Authorizing `identity:get_project(project_id=f5aedd2d5ee342aa80dfd96a062df0d9)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:42.142234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2da390f-896e-40b0-9319-e059b0b11d7a None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:42.144238 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 85/168] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:37:41 2022] GET /identity/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9 => generated 298 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:37:42.145693 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97301370-792e-47c5-9e27-57faf82b3fe7 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:42.146168 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97301370-792e-47c5-9e27-57faf82b3fe7 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:42.146631 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97301370-792e-47c5-9e27-57faf82b3fe7 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:42.148119 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 86/169] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:37:42.153588 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fbb020-2334-48bc-9fcd-f6fefbbea6eb None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:42.154125 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fbb020-2334-48bc-9fcd-f6fefbbea6eb None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:42.154895 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fbb020-2334-48bc-9fcd-f6fefbbea6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:42.200864 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-f2fbb020-2334-48bc-9fcd-f6fefbbea6eb None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:42.202086 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-f2fbb020-2334-48bc-9fcd-f6fefbbea6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:42.220410 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dce98e5-9f60-4a90-b7d4-5f5452072ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:42.221061 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dce98e5-9f60-4a90-b7d4-5f5452072ea0 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:42.221344 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dce98e5-9f60-4a90-b7d4-5f5452072ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:42.221616 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dce98e5-9f60-4a90-b7d4-5f5452072ea0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:42.232120 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:42.312704 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 87/170] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 12:37:42.314320 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fbb020-2334-48bc-9fcd-f6fefbbea6eb None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:42.350468 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dce98e5-9f60-4a90-b7d4-5f5452072ea0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:42.352105 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dce98e5-9f60-4a90-b7d4-5f5452072ea0 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:42.435540 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:42.437412 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fbb020-2334-48bc-9fcd-f6fefbbea6eb None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:42.476145 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4253cd3c-7dc6-4b18-8518-5c82097dd66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:42.476997 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253cd3c-7dc6-4b18-8518-5c82097dd66f None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:42.477264 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253cd3c-7dc6-4b18-8518-5c82097dd66f None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:42.477541 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253cd3c-7dc6-4b18-8518-5c82097dd66f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:42.478347 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4253cd3c-7dc6-4b18-8518-5c82097dd66f None admin] RBAC: Adding query filter params (role.id=6589779ddd834447a1ed1921e1c834f6, scope.project.id=f5aedd2d5ee342aa80dfd96a062df0d9, user.id=8fecfbe4ea8e4bd29589fe594487625c) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:42.478639 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4253cd3c-7dc6-4b18-8518-5c82097dd66f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:42.480131 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4253cd3c-7dc6-4b18-8518-5c82097dd66f None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:42.486533 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 88/171] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:37:42 2022] GET /identity/v3/role_assignments?user.id=8fecfbe4ea8e4bd29589fe594487625c&scope.project.id=f5aedd2d5ee342aa80dfd96a062df0d9&role.id=6589779ddd834447a1ed1921e1c834f6 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:42.487951 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163514d-e43c-454f-b7e0-e16e73940743 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:42.488241 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163514d-e43c-454f-b7e0-e16e73940743 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:42.488518 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163514d-e43c-454f-b7e0-e16e73940743 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:42.490074 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 89/172] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:37:42.495911 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d163514d-e43c-454f-b7e0-e16e73940743 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:42.507492 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 84/173] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:37:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 12:37:42.509663 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2100a8-0385-435b-bea4-320ea113338b None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:42.510153 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2100a8-0385-435b-bea4-320ea113338b None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:42.511027 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2100a8-0385-435b-bea4-320ea113338b None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:42.513011 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 85/174] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:37:42.514436 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546df564-4902-4413-94c6-46e29fee1fc4 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:42.514925 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546df564-4902-4413-94c6-46e29fee1fc4 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:42.515391 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546df564-4902-4413-94c6-46e29fee1fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:42.543822 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:42.556744 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-546df564-4902-4413-94c6-46e29fee1fc4 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:42.561506 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-546df564-4902-4413-94c6-46e29fee1fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:42.649168 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d163514d-e43c-454f-b7e0-e16e73940743 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:42.666110 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 86/175] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:37:42.667283 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546df564-4902-4413-94c6-46e29fee1fc4 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:42.691480 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e5eff8-8195-4894-8e5f-fe66019eb160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:42.692209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5eff8-8195-4894-8e5f-fe66019eb160 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:42.692493 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5eff8-8195-4894-8e5f-fe66019eb160 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:42.692825 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5eff8-8195-4894-8e5f-fe66019eb160 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:42.706176 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e5eff8-8195-4894-8e5f-fe66019eb160 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:37:42.712092 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-78e5eff8-8195-4894-8e5f-fe66019eb160 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 12:37:42.712092 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:37:42.712092 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:37:42.712092 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:37:42.712092 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:37:42.712092 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:37:42.712092 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:37:42.712092 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:37:42.712092 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:37:42.712092 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:37:42.712092 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:37:42.712092 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:37:42.712092 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:37:42.712092 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:37:42.712092 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:37:42.712092 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:37:42.712977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:37:42.712977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:37:42.712977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:37:42.712977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:37:42.712977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:37:42.712977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:37:42.712977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:37:42.712977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:37:42.712977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:37:42.712977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:37:42.712977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:37:42.712977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:37:42.712977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:37:42.712977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:37:42.712977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:37:42.712977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:37:42.713756 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:37:42.713756 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:37:42.713756 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:37:42.713756 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:37:42.713756 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:37:42.713756 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:37:42.713756 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:37:42.713756 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:37:42.713756 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:37:42.713756 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:37:42.713756 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 12:37:42.713756 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:37:42.713756 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 90/176] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:37:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:37:42.714401 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39741e-4dc2-42d6-9d24-44acd6f17136 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:42.714805 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39741e-4dc2-42d6-9d24-44acd6f17136 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:42.714805 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39741e-4dc2-42d6-9d24-44acd6f17136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:42.744341 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-ae39741e-4dc2-42d6-9d24-44acd6f17136 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:42.746079 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-ae39741e-4dc2-42d6-9d24-44acd6f17136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:42.785084 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:42.807618 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546df564-4902-4413-94c6-46e29fee1fc4 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:42.830962 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 91/177] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:37:42.837948 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3115bf5-6f75-4efd-8f2a-d8faf11cf8f5 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:42.838715 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3115bf5-6f75-4efd-8f2a-d8faf11cf8f5 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:42.839172 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3115bf5-6f75-4efd-8f2a-d8faf11cf8f5 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:42.841628 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 92/178] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:42 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:37:42.842505 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3115bf5-6f75-4efd-8f2a-d8faf11cf8f5 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:42.850135 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c6597-ae7b-4ddf-a78d-11a167bffb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:42.850879 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c6597-ae7b-4ddf-a78d-11a167bffb9f None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:42.851265 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c6597-ae7b-4ddf-a78d-11a167bffb9f None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:42.851653 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c6597-ae7b-4ddf-a78d-11a167bffb9f None admin] PATH_INFO: `/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:42.852922 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c6597-ae7b-4ddf-a78d-11a167bffb9f None admin] RBAC: Authorizing `identity:get_project(project_id=f5aedd2d5ee342aa80dfd96a062df0d9)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:42.854540 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c6597-ae7b-4ddf-a78d-11a167bffb9f None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:42.856414 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 87/179] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:37:42 2022] GET /identity/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:42.857124 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c6597-ae7b-4ddf-a78d-11a167bffb9f None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:42.858015 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:42.954491 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3115bf5-6f75-4efd-8f2a-d8faf11cf8f5 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:42.971143 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c6597-ae7b-4ddf-a78d-11a167bffb9f None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:42.991335 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993bbf3-2363-42ec-ba23-02907a700378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:42.992056 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993bbf3-2363-42ec-ba23-02907a700378 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:42.992337 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993bbf3-2363-42ec-ba23-02907a700378 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:42.992606 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993bbf3-2363-42ec-ba23-02907a700378 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:42.994148 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a993bbf3-2363-42ec-ba23-02907a700378 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:42.995599 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a993bbf3-2363-42ec-ba23-02907a700378 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:42.997366 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 93/180] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:37:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:42.998365 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d06d5-ec0c-4131-bbb2-40050880cb6b None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:42.998646 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d06d5-ec0c-4131-bbb2-40050880cb6b None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:42.998918 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d06d5-ec0c-4131-bbb2-40050880cb6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:43.008848 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71907e92-d2f2-4c1e-992a-3f9152a70dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:43.009471 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71907e92-d2f2-4c1e-992a-3f9152a70dc3 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:43.009754 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71907e92-d2f2-4c1e-992a-3f9152a70dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:43.010029 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71907e92-d2f2-4c1e-992a-3f9152a70dc3 None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:43.010746 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71907e92-d2f2-4c1e-992a-3f9152a70dc3 None admin] RBAC: Adding query filter params (name=member) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:43.011050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71907e92-d2f2-4c1e-992a-3f9152a70dc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:43.012468 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71907e92-d2f2-4c1e-992a-3f9152a70dc3 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:43.016885 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-515d06d5-ec0c-4131-bbb2-40050880cb6b None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:43.018065 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-515d06d5-ec0c-4131-bbb2-40050880cb6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:43.021027 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 88/181] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 26 12:37:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:43.021027 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71907e92-d2f2-4c1e-992a-3f9152a70dc3 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:43.037522 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:43.084784 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 94/182] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:37:43.095836 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515d06d5-ec0c-4131-bbb2-40050880cb6b None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:43.095929 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:43.138729 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71907e92-d2f2-4c1e-992a-3f9152a70dc3 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:43.173340 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8d4124-1a89-403a-81c8-56969fc7f0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:43.173978 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8d4124-1a89-403a-81c8-56969fc7f0e1 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:43.174253 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8d4124-1a89-403a-81c8-56969fc7f0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:43.174520 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8d4124-1a89-403a-81c8-56969fc7f0e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:43.185549 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8d4124-1a89-403a-81c8-56969fc7f0e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:37:43.189657 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515d06d5-ec0c-4131-bbb2-40050880cb6b None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:43.191579 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-ad8d4124-1a89-403a-81c8-56969fc7f0e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:37:43.191579 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:37:43.191579 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:37:43.191579 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:37:43.191579 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:37:43.191579 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:37:43.191579 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:37:43.191579 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:37:43.191579 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:37:43.191579 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:37:43.191579 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:37:43.191579 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:37:43.191579 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:37:43.191579 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:37:43.191579 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:37:43.191579 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:37:43.192486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:37:43.192486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:37:43.192486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:37:43.192486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:37:43.192486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:37:43.192486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:37:43.192486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:37:43.192486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:37:43.192486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:37:43.192486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:37:43.192486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:37:43.192486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:37:43.192486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:37:43.192486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:37:43.192486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:37:43.192486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:37:43.193512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:37:43.193512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:37:43.193512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:37:43.193512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:37:43.193512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:37:43.193512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:37:43.193512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:37:43.193512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:37:43.193512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:37:43.193512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:37:43.193512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:37:43.193512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:37:43.193512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 89/183] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 26 12:37:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:37:43.196885 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100022a5-897b-4b2c-a211-598f21785d89 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:43.197174 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100022a5-897b-4b2c-a211-598f21785d89 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:43.197447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100022a5-897b-4b2c-a211-598f21785d89 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:43.198859 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 90/184] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:37:43.199681 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100022a5-897b-4b2c-a211-598f21785d89 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:43.226086 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda40ebe-ff95-466f-83b1-9bc3dca622f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:43.226701 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda40ebe-ff95-466f-83b1-9bc3dca622f7 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:43.226979 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda40ebe-ff95-466f-83b1-9bc3dca622f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:43.227248 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda40ebe-ff95-466f-83b1-9bc3dca622f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:43.228132 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda40ebe-ff95-466f-83b1-9bc3dca622f7 None admin] RBAC: Adding query filter params (role.id=0e78444e43c744668178971af4087ca0, scope.project.id=f5aedd2d5ee342aa80dfd96a062df0d9, user.id=26e313f5057b4411a71a6bd67e68cb0e) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:43.228439 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda40ebe-ff95-466f-83b1-9bc3dca622f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:43.229878 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda40ebe-ff95-466f-83b1-9bc3dca622f7 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:43.235578 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 95/185] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:37:43 2022] GET /identity/v3/role_assignments?user.id=26e313f5057b4411a71a6bd67e68cb0e&scope.project.id=f5aedd2d5ee342aa80dfd96a062df0d9&role.id=0e78444e43c744668178971af4087ca0 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:43.236341 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda40ebe-ff95-466f-83b1-9bc3dca622f7 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:43.276652 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:43.316199 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100022a5-897b-4b2c-a211-598f21785d89 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:43.319885 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:43.354098 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda40ebe-ff95-466f-83b1-9bc3dca622f7 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:43.354518 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9255c7d-30b9-4a08-84e9-9447641ffd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:43.355158 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9255c7d-30b9-4a08-84e9-9447641ffd04 None admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:43.355441 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9255c7d-30b9-4a08-84e9-9447641ffd04 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:43.355753 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9255c7d-30b9-4a08-84e9-9447641ffd04 None admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:43.356461 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9255c7d-30b9-4a08-84e9-9447641ffd04 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:43.357995 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9255c7d-30b9-4a08-84e9-9447641ffd04 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:43.399697 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 91/186] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Jul 26 12:37:43 2022] POST /identity/v3/users => generated 297 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:37:43.399697 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9255c7d-30b9-4a08-84e9-9447641ffd04 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:43.412368 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6199e2ef-e7b0-41eb-a3ec-da5bb50b2547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:43.413039 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199e2ef-e7b0-41eb-a3ec-da5bb50b2547 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:43.413644 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199e2ef-e7b0-41eb-a3ec-da5bb50b2547 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:43.415156 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199e2ef-e7b0-41eb-a3ec-da5bb50b2547 None admin] PATH_INFO: `/v3/users/8fecfbe4ea8e4bd29589fe594487625c` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:43.416555 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6199e2ef-e7b0-41eb-a3ec-da5bb50b2547 None admin] RBAC: Authorizing `identity:get_user(user_id=8fecfbe4ea8e4bd29589fe594487625c)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:43.418074 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6199e2ef-e7b0-41eb-a3ec-da5bb50b2547 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:43.420231 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 96/187] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:37:43 2022] GET /identity/v3/users/8fecfbe4ea8e4bd29589fe594487625c => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:43.420865 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6199e2ef-e7b0-41eb-a3ec-da5bb50b2547 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:43.528282 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9255c7d-30b9-4a08-84e9-9447641ffd04 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:43.556919 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:43.564839 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6199e2ef-e7b0-41eb-a3ec-da5bb50b2547 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:43.568097 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4db45b9-0236-4d0d-a751-19efed098011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:43.568255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db45b9-0236-4d0d-a751-19efed098011 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:43.568512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db45b9-0236-4d0d-a751-19efed098011 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:43.569068 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db45b9-0236-4d0d-a751-19efed098011 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:43.584042 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4db45b9-0236-4d0d-a751-19efed098011 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:37:43.589150 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-d4db45b9-0236-4d0d-a751-19efed098011 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 12:37:43.589150 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:37:43.589150 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:37:43.589150 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:37:43.589150 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:37:43.589150 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:37:43.589150 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:37:43.589150 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:37:43.589150 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:37:43.589150 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:37:43.589150 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:37:43.589150 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:37:43.589150 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:37:43.589150 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:37:43.589150 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:37:43.589150 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:37:43.590284 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:37:43.590284 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:37:43.590284 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:37:43.590284 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:37:43.590284 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:37:43.590284 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:37:43.590284 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:37:43.590284 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:37:43.590284 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:37:43.590284 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:37:43.590284 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:37:43.590284 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:37:43.590284 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:37:43.590284 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:37:43.590284 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:37:43.590284 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:37:43.591159 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:37:43.591159 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:37:43.591159 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:37:43.591159 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:37:43.591159 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:37:43.591159 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:37:43.591159 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:37:43.591159 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:37:43.591159 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:37:43.591159 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:37:43.591159 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 12:37:43.591159 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:37:43.591159 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 92/188] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 26 12:37:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:37:43.592150 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa9a5f4-f59d-4681-899c-d846576a2c8c None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:43.592482 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa9a5f4-f59d-4681-899c-d846576a2c8c None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:43.592760 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa9a5f4-f59d-4681-899c-d846576a2c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:43.611698 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-2aa9a5f4-f59d-4681-899c-d846576a2c8c None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:43.612941 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-2aa9a5f4-f59d-4681-899c-d846576a2c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:43.622514 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1317d0-f642-4194-b466-7838d29610cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:43.623542 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1317d0-f642-4194-b466-7838d29610cc None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:43.623542 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1317d0-f642-4194-b466-7838d29610cc None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:43.623762 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1317d0-f642-4194-b466-7838d29610cc None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:43.624809 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1317d0-f642-4194-b466-7838d29610cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:43.624809 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1317d0-f642-4194-b466-7838d29610cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:43.626066 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1317d0-f642-4194-b466-7838d29610cc None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:43.637807 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 97/189] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:37:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:43.640626 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1317d0-f642-4194-b466-7838d29610cc None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:43.644029 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:43.702111 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 93/190] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:37:43.703324 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa9a5f4-f59d-4681-899c-d846576a2c8c None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:43.769793 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1317d0-f642-4194-b466-7838d29610cc None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:43.812821 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85afa96-7038-409b-b676-df8535f7bbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:43.813367 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85afa96-7038-409b-b676-df8535f7bbf2 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:43.813651 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85afa96-7038-409b-b676-df8535f7bbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:43.813925 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85afa96-7038-409b-b676-df8535f7bbf2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:43.833282 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa9a5f4-f59d-4681-899c-d846576a2c8c None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:43.851974 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:43.878179 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cc1372-2ba1-4295-b1e8-f3185ffbe193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:43.879591 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc1372-2ba1-4295-b1e8-f3185ffbe193 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:43.880231 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc1372-2ba1-4295-b1e8-f3185ffbe193 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:43.880781 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc1372-2ba1-4295-b1e8-f3185ffbe193 None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:43.881959 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cc1372-2ba1-4295-b1e8-f3185ffbe193 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:43.882493 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cc1372-2ba1-4295-b1e8-f3185ffbe193 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:43.884463 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cc1372-2ba1-4295-b1e8-f3185ffbe193 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:43.896392 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 94/191] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:37:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:43.898603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cc1372-2ba1-4295-b1e8-f3185ffbe193 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:43.908241 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85afa96-7038-409b-b676-df8535f7bbf2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:43.910499 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85afa96-7038-409b-b676-df8535f7bbf2 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:43.913242 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:44.011967 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cc1372-2ba1-4295-b1e8-f3185ffbe193 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:44.026698 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 98/192] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:37:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 12:37:44.027826 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85afa96-7038-409b-b676-df8535f7bbf2 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:44.060160 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05432bec-9285-45c8-a1d1-c5dba954f69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:44.061143 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05432bec-9285-45c8-a1d1-c5dba954f69b None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:44.061510 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05432bec-9285-45c8-a1d1-c5dba954f69b None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:44.061855 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05432bec-9285-45c8-a1d1-c5dba954f69b None admin] PATH_INFO: `/v3/users/60fc68e2ec8d474387db1dd3b8e0abdf` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:44.081538 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05432bec-9285-45c8-a1d1-c5dba954f69b None admin] RBAC: Authorizing `identity:get_user(user_id=60fc68e2ec8d474387db1dd3b8e0abdf)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:44.083042 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05432bec-9285-45c8-a1d1-c5dba954f69b None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:44.085543 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 95/193] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:37:43 2022] GET /identity/v3/users/60fc68e2ec8d474387db1dd3b8e0abdf => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:44.086411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05432bec-9285-45c8-a1d1-c5dba954f69b None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:44.151697 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85afa96-7038-409b-b676-df8535f7bbf2 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:44.153439 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:44.164022 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:44.194433 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f530912-dfc0-4b7e-8a8c-e56b04b4455c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:44.195162 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f530912-dfc0-4b7e-8a8c-e56b04b4455c None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:44.195516 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f530912-dfc0-4b7e-8a8c-e56b04b4455c None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:44.195894 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f530912-dfc0-4b7e-8a8c-e56b04b4455c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:44.207272 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f530912-dfc0-4b7e-8a8c-e56b04b4455c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:37:44.212479 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-2f530912-dfc0-4b7e-8a8c-e56b04b4455c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 12:37:44.212479 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:37:44.212479 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:37:44.212479 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:37:44.212479 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:37:44.212479 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:37:44.212479 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:37:44.212479 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:37:44.212479 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:37:44.212479 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:37:44.212479 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:37:44.212479 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:37:44.212479 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:37:44.212479 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:37:44.212479 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:37:44.212479 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:37:44.213341 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:37:44.213341 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:37:44.213341 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:37:44.213341 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:37:44.213341 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:37:44.213341 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:37:44.213341 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:37:44.213341 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:37:44.213341 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:37:44.213341 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:37:44.213341 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:37:44.213341 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:37:44.213341 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:37:44.213341 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:37:44.213341 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:37:44.213341 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:37:44.214154 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:37:44.214154 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:37:44.214154 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:37:44.214154 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:37:44.214154 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:37:44.214154 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:37:44.214154 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:37:44.214154 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:37:44.214154 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:37:44.214154 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:37:44.214154 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 12:37:44.214154 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:37:44.214154 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 99/194] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:37:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:37:44.214800 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f530912-dfc0-4b7e-8a8c-e56b04b4455c None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:44.224856 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05432bec-9285-45c8-a1d1-c5dba954f69b None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:44.264696 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eec5b1e-40d4-43c5-81fa-66aed2af5402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:44.265695 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec5b1e-40d4-43c5-81fa-66aed2af5402 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:44.266226 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec5b1e-40d4-43c5-81fa-66aed2af5402 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:44.266778 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec5b1e-40d4-43c5-81fa-66aed2af5402 None admin] PATH_INFO: `/v3/users/60fc68e2ec8d474387db1dd3b8e0abdf` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:44.268249 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eec5b1e-40d4-43c5-81fa-66aed2af5402 None admin] RBAC: Authorizing `identity:get_user(user_id=60fc68e2ec8d474387db1dd3b8e0abdf)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:44.270028 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eec5b1e-40d4-43c5-81fa-66aed2af5402 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:44.272141 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 96/195] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:37:44 2022] GET /identity/v3/users/60fc68e2ec8d474387db1dd3b8e0abdf => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:44.273379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eec5b1e-40d4-43c5-81fa-66aed2af5402 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:44.334302 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f530912-dfc0-4b7e-8a8c-e56b04b4455c None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:44.371607 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b938b547-9b8d-4eff-9331-498b77096ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:44.372413 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938b547-9b8d-4eff-9331-498b77096ae0 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:44.372768 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938b547-9b8d-4eff-9331-498b77096ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:44.373113 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938b547-9b8d-4eff-9331-498b77096ae0 None admin] PATH_INFO: `/v3/projects/cf365e8707b941b0b695d30a70dce1af` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:44.374674 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b938b547-9b8d-4eff-9331-498b77096ae0 None admin] RBAC: Authorizing `identity:get_project(project_id=cf365e8707b941b0b695d30a70dce1af)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:44.376235 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b938b547-9b8d-4eff-9331-498b77096ae0 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:44.378659 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 100/196] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:37:44 2022] GET /identity/v3/projects/cf365e8707b941b0b695d30a70dce1af => generated 312 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:44.384188 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b938b547-9b8d-4eff-9331-498b77096ae0 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:44.387658 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eec5b1e-40d4-43c5-81fa-66aed2af5402 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:44.390978 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:44.421118 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:44.427841 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd5c653-3e31-41e1-9334-89f5562df8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:44.428911 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5c653-3e31-41e1-9334-89f5562df8b6 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:44.429519 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5c653-3e31-41e1-9334-89f5562df8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:44.429970 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5c653-3e31-41e1-9334-89f5562df8b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:44.503006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b938b547-9b8d-4eff-9331-498b77096ae0 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:44.508883 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd5c653-3e31-41e1-9334-89f5562df8b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:44.510615 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd5c653-3e31-41e1-9334-89f5562df8b6 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:44.546819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe0d94-bce7-46c1-9625-624e63941224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:44.547541 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe0d94-bce7-46c1-9625-624e63941224 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:44.547929 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe0d94-bce7-46c1-9625-624e63941224 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:44.548290 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe0d94-bce7-46c1-9625-624e63941224 None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:44.549079 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe0d94-bce7-46c1-9625-624e63941224 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:44.549457 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe0d94-bce7-46c1-9625-624e63941224 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:44.551884 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe0d94-bce7-46c1-9625-624e63941224 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:44.561100 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 101/197] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 26 12:37:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:44.566533 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe0d94-bce7-46c1-9625-624e63941224 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:44.625167 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 97/198] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:37:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 12:37:44.626371 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd5c653-3e31-41e1-9334-89f5562df8b6 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:44.629918 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:44.669222 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:44.680891 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe0d94-bce7-46c1-9625-624e63941224 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:44.719910 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e0777d-4c0a-4720-ae4c-402a1af6d17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:44.720871 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0777d-4c0a-4720-ae4c-402a1af6d17a None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:44.721297 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0777d-4c0a-4720-ae4c-402a1af6d17a None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:44.721818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0777d-4c0a-4720-ae4c-402a1af6d17a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:44.742735 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd5c653-3e31-41e1-9334-89f5562df8b6 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:44.783262 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dce139e-748a-4264-abaa-6ba35414e1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:44.784250 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce139e-748a-4264-abaa-6ba35414e1c3 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:44.784532 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce139e-748a-4264-abaa-6ba35414e1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:44.784802 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce139e-748a-4264-abaa-6ba35414e1c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:44.785761 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dce139e-748a-4264-abaa-6ba35414e1c3 None admin] RBAC: Adding query filter params (role.id=01c9179cca2747a6b95b0decfca8b7ae, scope.project.id=cf365e8707b941b0b695d30a70dce1af, user.id=8fecfbe4ea8e4bd29589fe594487625c) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:44.786197 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dce139e-748a-4264-abaa-6ba35414e1c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:44.788057 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dce139e-748a-4264-abaa-6ba35414e1c3 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:44.795234 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 98/199] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:37:44 2022] GET /identity/v3/role_assignments?user.id=8fecfbe4ea8e4bd29589fe594487625c&scope.project.id=cf365e8707b941b0b695d30a70dce1af&role.id=01c9179cca2747a6b95b0decfca8b7ae => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:44.796333 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dce139e-748a-4264-abaa-6ba35414e1c3 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:44.803234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e0777d-4c0a-4720-ae4c-402a1af6d17a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:44.806064 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e0777d-4c0a-4720-ae4c-402a1af6d17a None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:44.873086 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:44.912302 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dce139e-748a-4264-abaa-6ba35414e1c3 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:44.916312 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:44.929651 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 102/200] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:37:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 12:37:44.930964 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e0777d-4c0a-4720-ae4c-402a1af6d17a None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:44.952420 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d6813e-bb09-4601-bbda-9cd230c00fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:44.953078 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6813e-bb09-4601-bbda-9cd230c00fda None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:44.953357 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6813e-bb09-4601-bbda-9cd230c00fda None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:44.953631 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6813e-bb09-4601-bbda-9cd230c00fda None admin] PATH_INFO: `/v3/users/b6ca251cd8f24a4fa7f5aa93cd0c29ac` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:44.954873 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d6813e-bb09-4601-bbda-9cd230c00fda None admin] RBAC: Authorizing `identity:get_user(user_id=b6ca251cd8f24a4fa7f5aa93cd0c29ac)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:44.956514 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d6813e-bb09-4601-bbda-9cd230c00fda None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:44.958194 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 99/201] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:37:44 2022] GET /identity/v3/users/b6ca251cd8f24a4fa7f5aa93cd0c29ac => generated 289 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:44.958968 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d6813e-bb09-4601-bbda-9cd230c00fda None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:45.044737 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e0777d-4c0a-4720-ae4c-402a1af6d17a None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:45.075056 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d6813e-bb09-4601-bbda-9cd230c00fda None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:45.085369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf9113-ec63-498d-859a-8e18c68bc7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:45.086533 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf9113-ec63-498d-859a-8e18c68bc7b8 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:45.087018 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf9113-ec63-498d-859a-8e18c68bc7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:45.087587 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf9113-ec63-498d-859a-8e18c68bc7b8 None admin] PATH_INFO: `/v3/projects/eb19926b01264c0a80580d9e18a4397d` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:45.089145 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bf9113-ec63-498d-859a-8e18c68bc7b8 None admin] RBAC: Authorizing `identity:get_project(project_id=eb19926b01264c0a80580d9e18a4397d)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:45.090938 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bf9113-ec63-498d-859a-8e18c68bc7b8 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:45.092941 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 103/202] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:37:44 2022] GET /identity/v3/projects/eb19926b01264c0a80580d9e18a4397d => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:45.094134 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf9113-ec63-498d-859a-8e18c68bc7b8 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:45.114803 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699100d-6481-46e9-8ba0-3d07dac397d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:45.115452 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699100d-6481-46e9-8ba0-3d07dac397d4 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:45.118237 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699100d-6481-46e9-8ba0-3d07dac397d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:45.118237 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699100d-6481-46e9-8ba0-3d07dac397d4 None admin] PATH_INFO: `/v3/projects/eb19926b01264c0a80580d9e18a4397d` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:45.118237 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f699100d-6481-46e9-8ba0-3d07dac397d4 None admin] RBAC: Authorizing `identity:get_project(project_id=eb19926b01264c0a80580d9e18a4397d)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:45.118783 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f699100d-6481-46e9-8ba0-3d07dac397d4 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:45.120688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 100/203] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:37:44 2022] GET /identity/v3/projects/eb19926b01264c0a80580d9e18a4397d => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:45.121491 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699100d-6481-46e9-8ba0-3d07dac397d4 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:45.125181 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:45.167467 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:45.217209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf9113-ec63-498d-859a-8e18c68bc7b8 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:45.263676 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699100d-6481-46e9-8ba0-3d07dac397d4 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:45.266387 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f01d72-f95d-43b5-8799-df36c9bcb82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:45.267290 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f01d72-f95d-43b5-8799-df36c9bcb82a None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:45.267856 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f01d72-f95d-43b5-8799-df36c9bcb82a None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:45.268336 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f01d72-f95d-43b5-8799-df36c9bcb82a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:45.307258 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5777abd-2bc4-47cf-9f45-8b145b882033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:45.307901 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5777abd-2bc4-47cf-9f45-8b145b882033 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:45.308177 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5777abd-2bc4-47cf-9f45-8b145b882033 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:45.308445 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5777abd-2bc4-47cf-9f45-8b145b882033 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:45.309490 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5777abd-2bc4-47cf-9f45-8b145b882033 None admin] RBAC: Adding query filter params (role.id=0e78444e43c744668178971af4087ca0, scope.project.id=eb19926b01264c0a80580d9e18a4397d, user.id=60fc68e2ec8d474387db1dd3b8e0abdf) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:45.309802 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5777abd-2bc4-47cf-9f45-8b145b882033 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:45.312172 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5777abd-2bc4-47cf-9f45-8b145b882033 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:45.319442 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 101/204] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:37:45 2022] GET /identity/v3/role_assignments?user.id=60fc68e2ec8d474387db1dd3b8e0abdf&scope.project.id=eb19926b01264c0a80580d9e18a4397d&role.id=0e78444e43c744668178971af4087ca0 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:45.320136 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5777abd-2bc4-47cf-9f45-8b145b882033 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:45.376562 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f01d72-f95d-43b5-8799-df36c9bcb82a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:45.378493 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f01d72-f95d-43b5-8799-df36c9bcb82a None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:45.476747 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:45.514684 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:45.536445 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5777abd-2bc4-47cf-9f45-8b145b882033 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:45.586320 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 104/205] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:37:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 12:37:45.591939 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f01d72-f95d-43b5-8799-df36c9bcb82a None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:45.603526 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d19da-9a42-400c-984a-c33d694d10f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:45.604181 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d19da-9a42-400c-984a-c33d694d10f7 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:45.604458 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d19da-9a42-400c-984a-c33d694d10f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:45.604727 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d19da-9a42-400c-984a-c33d694d10f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:45.605520 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113d19da-9a42-400c-984a-c33d694d10f7 None admin] RBAC: Adding query filter params (role.id=6589779ddd834447a1ed1921e1c834f6, scope.project.id=eb19926b01264c0a80580d9e18a4397d, user.id=60fc68e2ec8d474387db1dd3b8e0abdf) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:45.605823 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113d19da-9a42-400c-984a-c33d694d10f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:45.607297 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113d19da-9a42-400c-984a-c33d694d10f7 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:45.616309 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 102/206] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:37:45 2022] GET /identity/v3/role_assignments?user.id=60fc68e2ec8d474387db1dd3b8e0abdf&scope.project.id=eb19926b01264c0a80580d9e18a4397d&role.id=6589779ddd834447a1ed1921e1c834f6 => generated 259 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:45.656714 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa960fb4-d680-47f2-8ce3-aaf79f0779f4 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:45.657016 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa960fb4-d680-47f2-8ce3-aaf79f0779f4 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:45.657405 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa960fb4-d680-47f2-8ce3-aaf79f0779f4 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:45.663669 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 103/207] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 12:37:45.665553 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3951232-321d-4995-80df-ccec0f82a50b None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:45.666076 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3951232-321d-4995-80df-ccec0f82a50b None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:45.666570 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3951232-321d-4995-80df-ccec0f82a50b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:45.687080 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-d3951232-321d-4995-80df-ccec0f82a50b None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:45.688841 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-d3951232-321d-4995-80df-ccec0f82a50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:45.750823 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f01d72-f95d-43b5-8799-df36c9bcb82a None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:45.791424 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae643d-c3cf-4b9a-b921-61d14e7d0790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:45.792234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae643d-c3cf-4b9a-b921-61d14e7d0790 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:45.792518 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae643d-c3cf-4b9a-b921-61d14e7d0790 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:45.792797 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae643d-c3cf-4b9a-b921-61d14e7d0790 None admin] PATH_INFO: `/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:45.794239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ae643d-c3cf-4b9a-b921-61d14e7d0790 None admin] RBAC: Authorizing `identity:get_project(project_id=f5aedd2d5ee342aa80dfd96a062df0d9)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:45.795741 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ae643d-c3cf-4b9a-b921-61d14e7d0790 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:45.798908 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 105/208] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:37:45 2022] GET /identity/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:37:45.803924 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae643d-c3cf-4b9a-b921-61d14e7d0790 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:45.842134 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 104/209] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:37:45.848012 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:45.872262 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3951232-321d-4995-80df-ccec0f82a50b None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:45.943941 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:45.961989 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae643d-c3cf-4b9a-b921-61d14e7d0790 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:45.985826 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3951232-321d-4995-80df-ccec0f82a50b None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:46.008902 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7662d992-7c6e-4ba7-b447-98ee5575c1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:46.009537 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7662d992-7c6e-4ba7-b447-98ee5575c1ae None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:46.010162 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7662d992-7c6e-4ba7-b447-98ee5575c1ae None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:46.010520 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7662d992-7c6e-4ba7-b447-98ee5575c1ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:46.011663 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7662d992-7c6e-4ba7-b447-98ee5575c1ae None admin] RBAC: Adding query filter params (role.id=db16c43424694218852e80f51fb1ee70, scope.project.id=f5aedd2d5ee342aa80dfd96a062df0d9, user.id=b6ca251cd8f24a4fa7f5aa93cd0c29ac) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:46.012496 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7662d992-7c6e-4ba7-b447-98ee5575c1ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:46.014543 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7662d992-7c6e-4ba7-b447-98ee5575c1ae None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:46.023310 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2072db-1c12-490a-839b-efd094a45d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:46.024069 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2072db-1c12-490a-839b-efd094a45d50 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:46.024416 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2072db-1c12-490a-839b-efd094a45d50 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:46.024753 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2072db-1c12-490a-839b-efd094a45d50 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:46.027785 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 106/210] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:37:45 2022] GET /identity/v3/role_assignments?user.id=b6ca251cd8f24a4fa7f5aa93cd0c29ac&scope.project.id=f5aedd2d5ee342aa80dfd96a062df0d9&role.id=db16c43424694218852e80f51fb1ee70 => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:46.030176 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d6281-1e65-4eeb-ad09-941f12f3522a None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:46.030456 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d6281-1e65-4eeb-ad09-941f12f3522a None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:46.030726 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d6281-1e65-4eeb-ad09-941f12f3522a None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:46.040847 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 107/211] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:46 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:37:46.043888 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053d106-b947-4ef3-ac35-ea6fc073c869 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:46.044192 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053d106-b947-4ef3-ac35-ea6fc073c869 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:46.044463 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053d106-b947-4ef3-ac35-ea6fc073c869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:46.055222 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2072db-1c12-490a-839b-efd094a45d50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:37:46.061585 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-ff2072db-1c12-490a-839b-efd094a45d50 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 12:37:46.061585 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:37:46.061585 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:37:46.061585 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:37:46.061585 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:37:46.061585 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:37:46.061585 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:37:46.061585 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:37:46.061585 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:37:46.061585 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:37:46.061585 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:37:46.061585 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:37:46.061585 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:37:46.061585 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:37:46.061585 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:37:46.061585 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:37:46.062560 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:37:46.062560 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:37:46.062560 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:37:46.062560 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:37:46.062560 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:37:46.062560 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:37:46.062560 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:37:46.062560 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:37:46.062560 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:37:46.062560 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:37:46.062560 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:37:46.062560 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:37:46.062560 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:37:46.062560 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:37:46.062560 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:37:46.062560 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:37:46.064633 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:37:46.064633 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:37:46.064633 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:37:46.064633 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:37:46.064633 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:37:46.064633 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:37:46.064633 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:37:46.064633 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:37:46.064633 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:37:46.064633 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:37:46.064633 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 12:37:46.064633 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:37:46.064633 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 105/212] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:37:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:37:46.072070 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2072db-1c12-490a-839b-efd094a45d50 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:46.084764 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-b053d106-b947-4ef3-ac35-ea6fc073c869 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:46.088239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-b053d106-b947-4ef3-ac35-ea6fc073c869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:46.187083 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 108/213] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:37:46.232294 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053d106-b947-4ef3-ac35-ea6fc073c869 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:46.258730 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2072db-1c12-490a-839b-efd094a45d50 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:46.264662 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:46.291863 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:46.306584 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7355d2c-6c9f-484a-b0be-7437f4b56341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:46.307367 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7355d2c-6c9f-484a-b0be-7437f4b56341 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:46.307797 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7355d2c-6c9f-484a-b0be-7437f4b56341 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:46.308176 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7355d2c-6c9f-484a-b0be-7437f4b56341 None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:46.309126 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7355d2c-6c9f-484a-b0be-7437f4b56341 None admin] RBAC: Adding query filter params (name=member) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:46.309533 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7355d2c-6c9f-484a-b0be-7437f4b56341 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:46.311307 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7355d2c-6c9f-484a-b0be-7437f4b56341 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:46.324739 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 106/214] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 26 12:37:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:46.330728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7355d2c-6c9f-484a-b0be-7437f4b56341 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:46.373159 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053d106-b947-4ef3-ac35-ea6fc073c869 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:46.421341 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4a32fd-0a7a-474d-9949-2d6fc9167b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:46.422100 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a32fd-0a7a-474d-9949-2d6fc9167b9e None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:46.422330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a32fd-0a7a-474d-9949-2d6fc9167b9e None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:46.422766 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a32fd-0a7a-474d-9949-2d6fc9167b9e None admin] PATH_INFO: `/v3/domains/default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:46.424902 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4a32fd-0a7a-474d-9949-2d6fc9167b9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:46.426674 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4a32fd-0a7a-474d-9949-2d6fc9167b9e None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:46.428894 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 109/215] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:37:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:37:46.435560 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4a32fd-0a7a-474d-9949-2d6fc9167b9e None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:46.492845 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7355d2c-6c9f-484a-b0be-7437f4b56341 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:46.537661 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addaf975-4bb7-4508-982f-c4ec26b8eadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:46.538229 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addaf975-4bb7-4508-982f-c4ec26b8eadb None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:46.539301 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addaf975-4bb7-4508-982f-c4ec26b8eadb None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:46.539301 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addaf975-4bb7-4508-982f-c4ec26b8eadb None admin] PATH_INFO: `/v3/users/bc90308d618148e49910a19ad2640ab1` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:46.560813 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addaf975-4bb7-4508-982f-c4ec26b8eadb None admin] RBAC: Authorizing `identity:get_user(user_id=bc90308d618148e49910a19ad2640ab1)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:46.562620 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addaf975-4bb7-4508-982f-c4ec26b8eadb None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:46.564585 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 107/216] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:37:46 2022] GET /identity/v3/users/bc90308d618148e49910a19ad2640ab1 => generated 297 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:37:46.565622 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:46.570545 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addaf975-4bb7-4508-982f-c4ec26b8eadb None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:46.578740 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4a32fd-0a7a-474d-9949-2d6fc9167b9e None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:46.617770 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e1ca3-9266-4c80-b65e-74f66704f9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:46.618472 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e1ca3-9266-4c80-b65e-74f66704f9f4 None admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:46.618754 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e1ca3-9266-4c80-b65e-74f66704f9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:46.619077 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e1ca3-9266-4c80-b65e-74f66704f9f4 None admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:46.619820 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e1ca3-9266-4c80-b65e-74f66704f9f4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:46.621268 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e1ca3-9266-4c80-b65e-74f66704f9f4 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:46.626337 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:46.648712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 110/217] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Jul 26 12:37:46 2022] POST /identity/v3/users => generated 297 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:37:46.699202 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addaf975-4bb7-4508-982f-c4ec26b8eadb None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:46.743955 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fbbbd5-0ded-4684-93f4-70294066916c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:46.744586 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fbbbd5-0ded-4684-93f4-70294066916c None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:46.744862 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fbbbd5-0ded-4684-93f4-70294066916c None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:46.745132 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fbbbd5-0ded-4684-93f4-70294066916c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:46.839171 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fbbbd5-0ded-4684-93f4-70294066916c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:46.840602 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fbbbd5-0ded-4684-93f4-70294066916c None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:46.891820 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:46.964822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 108/218] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:37:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 12:37:46.970381 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e1ca3-9266-4c80-b65e-74f66704f9f4 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:47.081954 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e1ca3-9266-4c80-b65e-74f66704f9f4 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:47.128418 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f367cf7-4aee-4e9f-8a14-ca26387e940a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:47.129503 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f367cf7-4aee-4e9f-8a14-ca26387e940a None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:47.129984 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f367cf7-4aee-4e9f-8a14-ca26387e940a None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:47.130460 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f367cf7-4aee-4e9f-8a14-ca26387e940a None admin] PATH_INFO: `/v3/projects/eb19926b01264c0a80580d9e18a4397d` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:47.131902 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f367cf7-4aee-4e9f-8a14-ca26387e940a None admin] RBAC: Authorizing `identity:get_project(project_id=eb19926b01264c0a80580d9e18a4397d)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:47.133645 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f367cf7-4aee-4e9f-8a14-ca26387e940a None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:47.135525 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 111/219] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:37:46 2022] GET /identity/v3/projects/eb19926b01264c0a80580d9e18a4397d => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:47.141775 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fbbbd5-0ded-4684-93f4-70294066916c None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:47.283864 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fbbbd5-0ded-4684-93f4-70294066916c None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:47.334229 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a0a85-9d3d-4f20-8589-2f04f7020b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:47.334851 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a0a85-9d3d-4f20-8589-2f04f7020b64 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:47.335128 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a0a85-9d3d-4f20-8589-2f04f7020b64 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:47.335551 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a0a85-9d3d-4f20-8589-2f04f7020b64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:47.336378 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2a0a85-9d3d-4f20-8589-2f04f7020b64 None admin] RBAC: Adding query filter params (role.id=01c9179cca2747a6b95b0decfca8b7ae, scope.project.id=eb19926b01264c0a80580d9e18a4397d, user.id=bc90308d618148e49910a19ad2640ab1) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:47.336681 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2a0a85-9d3d-4f20-8589-2f04f7020b64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:47.338177 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2a0a85-9d3d-4f20-8589-2f04f7020b64 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:47.348225 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 109/220] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:37:47 2022] GET /identity/v3/role_assignments?user.id=bc90308d618148e49910a19ad2640ab1&scope.project.id=eb19926b01264c0a80580d9e18a4397d&role.id=01c9179cca2747a6b95b0decfca8b7ae => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:47.750715 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc5b3e-9e54-462b-88e5-dee1ed03e704 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:47.751760 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc5b3e-9e54-462b-88e5-dee1ed03e704 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:47.752285 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc5b3e-9e54-462b-88e5-dee1ed03e704 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:47.753857 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 112/221] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:37:47.760516 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181cb1c-c2f4-4dae-a347-54ffab5f1add None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:47.760798 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181cb1c-c2f4-4dae-a347-54ffab5f1add None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:47.761077 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181cb1c-c2f4-4dae-a347-54ffab5f1add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:47.777797 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:47.784119 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-1181cb1c-c2f4-4dae-a347-54ffab5f1add None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:47.785306 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-1181cb1c-c2f4-4dae-a347-54ffab5f1add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:47.874558 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 110/222] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:37:47.908982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbc5b3e-9e54-462b-88e5-dee1ed03e704 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:47.932231 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:48.110657 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbc5b3e-9e54-462b-88e5-dee1ed03e704 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:48.122691 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200165ca-c3c8-4307-8294-b89f5b4f2bbf None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:48.122993 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200165ca-c3c8-4307-8294-b89f5b4f2bbf None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:48.123244 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200165ca-c3c8-4307-8294-b89f5b4f2bbf None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:48.125481 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 111/223] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:37:48.131625 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f3ef1-254a-46aa-a735-1851647e5d7b None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:48.131944 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f3ef1-254a-46aa-a735-1851647e5d7b None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:48.132754 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f3ef1-254a-46aa-a735-1851647e5d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:48.168814 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-ef7f3ef1-254a-46aa-a735-1851647e5d7b None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:48.170049 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-ef7f3ef1-254a-46aa-a735-1851647e5d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:48.176629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b6925a-8067-4735-8975-bf90f0e2caf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:48.177289 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6925a-8067-4735-8975-bf90f0e2caf1 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:48.177567 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6925a-8067-4735-8975-bf90f0e2caf1 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:48.177840 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6925a-8067-4735-8975-bf90f0e2caf1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:48.199983 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b6925a-8067-4735-8975-bf90f0e2caf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:37:48.205417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-f3b6925a-8067-4735-8975-bf90f0e2caf1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:37:48.205417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:37:48.205417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:37:48.205417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:37:48.205417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:37:48.205417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:37:48.205417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:37:48.205417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:37:48.205417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:37:48.205417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:37:48.205417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:37:48.205417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:37:48.205417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:37:48.205417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:37:48.205417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:37:48.205417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:37:48.206296 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:37:48.206296 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:37:48.206296 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:37:48.206296 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:37:48.206296 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:37:48.206296 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:37:48.206296 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:37:48.206296 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:37:48.206296 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:37:48.206296 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:37:48.206296 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:37:48.206296 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:37:48.206296 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:37:48.206296 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:37:48.206296 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:37:48.206296 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:37:48.207081 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:37:48.207081 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:37:48.207081 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:37:48.207081 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:37:48.207081 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:37:48.207081 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:37:48.207081 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:37:48.207081 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:37:48.207081 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:37:48.207081 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:37:48.207081 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:37:48.207081 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:37:48.217377 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 113/224] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 26 12:37:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:37:48.218284 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b6925a-8067-4735-8975-bf90f0e2caf1 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:48.318586 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 112/225] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 12:37:48.367783 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:48.379436 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b6925a-8067-4735-8975-bf90f0e2caf1 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:48.384000 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7f3ef1-254a-46aa-a735-1851647e5d7b None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:48.445943 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19937e48-6a7f-4c6c-a81b-612b63ee9992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:48.446576 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19937e48-6a7f-4c6c-a81b-612b63ee9992 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:48.446857 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19937e48-6a7f-4c6c-a81b-612b63ee9992 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:48.447133 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19937e48-6a7f-4c6c-a81b-612b63ee9992 None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:48.447888 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19937e48-6a7f-4c6c-a81b-612b63ee9992 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:48.448196 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19937e48-6a7f-4c6c-a81b-612b63ee9992 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:48.449582 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19937e48-6a7f-4c6c-a81b-612b63ee9992 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:48.467926 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 114/226] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:37:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:48.473681 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19937e48-6a7f-4c6c-a81b-612b63ee9992 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:48.516019 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:48.593209 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7f3ef1-254a-46aa-a735-1851647e5d7b None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:48.624225 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19937e48-6a7f-4c6c-a81b-612b63ee9992 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:48.669190 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c639c31f-7975-4a44-9675-412499de1cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:48.670208 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639c31f-7975-4a44-9675-412499de1cf4 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:48.670710 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639c31f-7975-4a44-9675-412499de1cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:48.671183 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639c31f-7975-4a44-9675-412499de1cf4 None admin] PATH_INFO: `/v3/users/60fc68e2ec8d474387db1dd3b8e0abdf` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:48.672782 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c639c31f-7975-4a44-9675-412499de1cf4 None admin] RBAC: Authorizing `identity:get_user(user_id=60fc68e2ec8d474387db1dd3b8e0abdf)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:48.672889 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9ab304-59aa-4ac3-9248-ce816025a5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:48.673549 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ab304-59aa-4ac3-9248-ce816025a5e2 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:48.673835 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ab304-59aa-4ac3-9248-ce816025a5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:48.674115 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ab304-59aa-4ac3-9248-ce816025a5e2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:48.675280 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c639c31f-7975-4a44-9675-412499de1cf4 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:48.677728 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 115/227] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:37:48 2022] GET /identity/v3/users/60fc68e2ec8d474387db1dd3b8e0abdf => generated 283 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:37:48.684601 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c639c31f-7975-4a44-9675-412499de1cf4 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:48.688776 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:48.707490 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9ab304-59aa-4ac3-9248-ce816025a5e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:37:48.716326 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-be9ab304-59aa-4ac3-9248-ce816025a5e2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 12:37:48.716326 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:37:48.716326 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:37:48.716326 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:37:48.716326 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:37:48.716326 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:37:48.716326 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:37:48.716326 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:37:48.716326 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:37:48.716326 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:37:48.716326 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:37:48.716326 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:37:48.716326 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:37:48.716326 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:37:48.716326 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:37:48.716326 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:37:48.717395 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:37:48.717395 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:37:48.717395 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:37:48.717395 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:37:48.717395 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:37:48.717395 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:37:48.717395 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:37:48.717395 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:37:48.717395 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:37:48.717395 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:37:48.717395 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:37:48.717395 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:37:48.717395 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:37:48.717395 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:37:48.717395 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:37:48.717395 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:37:48.718180 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:37:48.718180 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:37:48.718180 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:37:48.718180 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:37:48.718180 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:37:48.718180 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:37:48.718180 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:37:48.718180 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:37:48.718180 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:37:48.718180 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:37:48.718180 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 12:37:48.718180 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:37:48.723346 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 113/228] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 26 12:37:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:37:48.727097 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9ab304-59aa-4ac3-9248-ce816025a5e2 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:48.827674 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c639c31f-7975-4a44-9675-412499de1cf4 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:48.867613 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9ab304-59aa-4ac3-9248-ce816025a5e2 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:48.877480 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bba431-9ded-460a-bb80-6f72d98e6707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:48.878316 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bba431-9ded-460a-bb80-6f72d98e6707 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:48.878600 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bba431-9ded-460a-bb80-6f72d98e6707 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:48.878875 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bba431-9ded-460a-bb80-6f72d98e6707 None admin] PATH_INFO: `/v3/projects/eb19926b01264c0a80580d9e18a4397d` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:48.880141 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bba431-9ded-460a-bb80-6f72d98e6707 None admin] RBAC: Authorizing `identity:get_project(project_id=eb19926b01264c0a80580d9e18a4397d)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:48.881654 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bba431-9ded-460a-bb80-6f72d98e6707 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:48.883900 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 116/229] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:37:48 2022] GET /identity/v3/projects/eb19926b01264c0a80580d9e18a4397d => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:48.892615 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bba431-9ded-460a-bb80-6f72d98e6707 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:48.909599 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:48.945390 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1584696b-f0f5-463f-94dd-4946693f1fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:48.946039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1584696b-f0f5-463f-94dd-4946693f1fd4 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:48.946325 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1584696b-f0f5-463f-94dd-4946693f1fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:48.946599 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1584696b-f0f5-463f-94dd-4946693f1fd4 None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:48.947341 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1584696b-f0f5-463f-94dd-4946693f1fd4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:48.947663 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1584696b-f0f5-463f-94dd-4946693f1fd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:48.949080 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1584696b-f0f5-463f-94dd-4946693f1fd4 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:48.972606 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 114/230] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:37:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:48.978952 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1584696b-f0f5-463f-94dd-4946693f1fd4 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:49.019050 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:49.110467 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1584696b-f0f5-463f-94dd-4946693f1fd4 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:49.130754 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bba431-9ded-460a-bb80-6f72d98e6707 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:49.154412 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0398eeff-2e91-42d5-93a5-4322cd05c786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:49.155450 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398eeff-2e91-42d5-93a5-4322cd05c786 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:49.156342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398eeff-2e91-42d5-93a5-4322cd05c786 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:49.157043 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398eeff-2e91-42d5-93a5-4322cd05c786 None admin] PATH_INFO: `/v3/users/60fc68e2ec8d474387db1dd3b8e0abdf` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:49.158621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0398eeff-2e91-42d5-93a5-4322cd05c786 None admin] RBAC: Authorizing `identity:get_user(user_id=60fc68e2ec8d474387db1dd3b8e0abdf)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:49.162651 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0398eeff-2e91-42d5-93a5-4322cd05c786 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:49.162651 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 115/231] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:37:48 2022] GET /identity/v3/users/60fc68e2ec8d474387db1dd3b8e0abdf => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:49.164211 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97439ad-f343-4aaf-bb6f-0dc89c896855 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:49.164619 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97439ad-f343-4aaf-bb6f-0dc89c896855 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:49.164999 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97439ad-f343-4aaf-bb6f-0dc89c896855 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:49.166505 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 116/232] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:37:49.168583 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97439ad-f343-4aaf-bb6f-0dc89c896855 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:49.173733 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c18f2c-d24b-48e8-88a2-3642828123f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:49.174460 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c18f2c-d24b-48e8-88a2-3642828123f7 None admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:49.174752 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c18f2c-d24b-48e8-88a2-3642828123f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:49.175026 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c18f2c-d24b-48e8-88a2-3642828123f7 None admin] PATH_INFO: `/v3/projects/eb19926b01264c0a80580d9e18a4397d/users/60fc68e2ec8d474387db1dd3b8e0abdf/roles/0e78444e43c744668178971af4087ca0` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:49.182830 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c18f2c-d24b-48e8-88a2-3642828123f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb19926b01264c0a80580d9e18a4397d, user_id=60fc68e2ec8d474387db1dd3b8e0abdf, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:49.184684 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c18f2c-d24b-48e8-88a2-3642828123f7 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:49.201044 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 117/233] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 26 12:37:48 2022] PUT /identity/v3/projects/eb19926b01264c0a80580d9e18a4397d/users/60fc68e2ec8d474387db1dd3b8e0abdf/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:37:49.203135 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2247b3-ef88-4d18-b79c-5fa379b8b7be None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:49.203989 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2247b3-ef88-4d18-b79c-5fa379b8b7be None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:49.204277 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2247b3-ef88-4d18-b79c-5fa379b8b7be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:49.228189 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-5f2247b3-ef88-4d18-b79c-5fa379b8b7be None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:49.230569 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-5f2247b3-ef88-4d18-b79c-5fa379b8b7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:49.256090 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:49.317557 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97439ad-f343-4aaf-bb6f-0dc89c896855 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:49.328209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 118/234] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:37:49.329368 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e4927-010f-45b7-bed1-e7cf3f8af19c None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:49.329657 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e4927-010f-45b7-bed1-e7cf3f8af19c None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:49.329928 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e4927-010f-45b7-bed1-e7cf3f8af19c None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:49.331360 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 119/235] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:37:49.338435 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0630e0bf-6219-469a-ad1d-aef42b0a1420 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:49.338863 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0630e0bf-6219-469a-ad1d-aef42b0a1420 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:49.339236 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0630e0bf-6219-469a-ad1d-aef42b0a1420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:49.363552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-0630e0bf-6219-469a-ad1d-aef42b0a1420 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:49.365153 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-0630e0bf-6219-469a-ad1d-aef42b0a1420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:49.366883 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9056b211-c192-4946-9fd6-835feeb2dec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:49.367857 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056b211-c192-4946-9fd6-835feeb2dec3 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:49.368145 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056b211-c192-4946-9fd6-835feeb2dec3 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:49.368423 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056b211-c192-4946-9fd6-835feeb2dec3 None admin] PATH_INFO: `/v3/projects/eb19926b01264c0a80580d9e18a4397d` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:49.370646 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9056b211-c192-4946-9fd6-835feeb2dec3 None admin] RBAC: Authorizing `identity:get_project(project_id=eb19926b01264c0a80580d9e18a4397d)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:49.372208 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9056b211-c192-4946-9fd6-835feeb2dec3 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:49.373955 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 117/236] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:37:49 2022] GET /identity/v3/projects/eb19926b01264c0a80580d9e18a4397d => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:49.380039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9056b211-c192-4946-9fd6-835feeb2dec3 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:49.447626 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:49.463375 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 120/237] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:37:49.464258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0630e0bf-6219-469a-ad1d-aef42b0a1420 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:49.511601 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9056b211-c192-4946-9fd6-835feeb2dec3 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:49.539858 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:49.553580 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42245831-86e6-4e0a-abe7-bf0078ef0c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:49.554778 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42245831-86e6-4e0a-abe7-bf0078ef0c61 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:49.555242 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42245831-86e6-4e0a-abe7-bf0078ef0c61 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:49.556084 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42245831-86e6-4e0a-abe7-bf0078ef0c61 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:49.568404 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42245831-86e6-4e0a-abe7-bf0078ef0c61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:37:49.574411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-42245831-86e6-4e0a-abe7-bf0078ef0c61 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 12:37:49.574411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:37:49.574411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:37:49.574411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:37:49.574411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:37:49.574411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:37:49.574411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:37:49.574411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:37:49.574411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:37:49.574411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:37:49.574411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:37:49.574411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:37:49.574411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:37:49.574411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:37:49.574411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:37:49.574411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:37:49.575333 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:37:49.575333 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:37:49.575333 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:37:49.575333 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:37:49.575333 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:37:49.575333 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:37:49.575333 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:37:49.575333 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:37:49.575333 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:37:49.575333 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:37:49.575333 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:37:49.575333 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:37:49.575333 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:37:49.575333 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:37:49.575333 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:37:49.575333 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:37:49.576127 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:37:49.576127 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:37:49.576127 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:37:49.576127 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:37:49.576127 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:37:49.576127 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:37:49.576127 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:37:49.576127 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:37:49.576127 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:37:49.576127 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:37:49.576127 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 12:37:49.576127 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:37:49.576127 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 118/238] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:37:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:37:49.577258 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42245831-86e6-4e0a-abe7-bf0078ef0c61 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:49.605201 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0630e0bf-6219-469a-ad1d-aef42b0a1420 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:49.645029 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc5b7b8-631f-40e1-a2ef-26606f28ad37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:49.645734 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc5b7b8-631f-40e1-a2ef-26606f28ad37 None admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:49.646015 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc5b7b8-631f-40e1-a2ef-26606f28ad37 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:49.646290 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc5b7b8-631f-40e1-a2ef-26606f28ad37 None admin] PATH_INFO: `/v3/projects/eb19926b01264c0a80580d9e18a4397d/users/60fc68e2ec8d474387db1dd3b8e0abdf/roles/6589779ddd834447a1ed1921e1c834f6` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:49.648683 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc5b7b8-631f-40e1-a2ef-26606f28ad37 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb19926b01264c0a80580d9e18a4397d, user_id=60fc68e2ec8d474387db1dd3b8e0abdf, role_id=6589779ddd834447a1ed1921e1c834f6)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:49.650049 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc5b7b8-631f-40e1-a2ef-26606f28ad37 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:49.668543 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 121/239] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 26 12:37:49 2022] PUT /identity/v3/projects/eb19926b01264c0a80580d9e18a4397d/users/60fc68e2ec8d474387db1dd3b8e0abdf/roles/6589779ddd834447a1ed1921e1c834f6 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:37:49.670389 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc5b7b8-631f-40e1-a2ef-26606f28ad37 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:49.708769 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42245831-86e6-4e0a-abe7-bf0078ef0c61 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:49.748167 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e01c0-3a42-4d33-9e99-8ab8078395d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:49.749080 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e01c0-3a42-4d33-9e99-8ab8078395d0 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:49.749369 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e01c0-3a42-4d33-9e99-8ab8078395d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:49.749821 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e01c0-3a42-4d33-9e99-8ab8078395d0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:49.751366 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069e01c0-3a42-4d33-9e99-8ab8078395d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:49.753119 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:49.754587 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069e01c0-3a42-4d33-9e99-8ab8078395d0 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:49.758560 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 119/240] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:37:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:49.769695 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e01c0-3a42-4d33-9e99-8ab8078395d0 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:49.811819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc5b7b8-631f-40e1-a2ef-26606f28ad37 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:49.861563 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:49.888472 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf19a1-bde6-4554-b550-bc71ea31511d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:49.889124 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf19a1-bde6-4554-b550-bc71ea31511d None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:49.889392 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf19a1-bde6-4554-b550-bc71ea31511d None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:49.889667 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf19a1-bde6-4554-b550-bc71ea31511d None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:49.890413 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abf19a1-bde6-4554-b550-bc71ea31511d None admin] RBAC: Adding query filter params (name=reader) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:49.890736 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abf19a1-bde6-4554-b550-bc71ea31511d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:49.892182 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abf19a1-bde6-4554-b550-bc71ea31511d None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:49.907055 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 122/241] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 26 12:37:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:49.918219 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf19a1-bde6-4554-b550-bc71ea31511d None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:49.930045 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e01c0-3a42-4d33-9e99-8ab8078395d0 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:49.975630 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e50706-325d-43a9-b2bd-79a112eccaa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:49.976734 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e50706-325d-43a9-b2bd-79a112eccaa9 None admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:49.977246 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e50706-325d-43a9-b2bd-79a112eccaa9 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:49.977628 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e50706-325d-43a9-b2bd-79a112eccaa9 None admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:49.978427 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e50706-325d-43a9-b2bd-79a112eccaa9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:49.980017 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e50706-325d-43a9-b2bd-79a112eccaa9 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:50.006344 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 120/242] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Jul 26 12:37:49 2022] POST /identity/v3/users => generated 293 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:37:50.008079 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6634d01c-2ae8-4a26-ac71-0bd9b2f09bba None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:50.008474 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6634d01c-2ae8-4a26-ac71-0bd9b2f09bba None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:50.008854 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6634d01c-2ae8-4a26-ac71-0bd9b2f09bba None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:50.015285 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 121/243] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:37:50.021424 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f6666-b957-450b-8f4d-602c9ffb375c None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:50.021855 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f6666-b957-450b-8f4d-602c9ffb375c None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:50.022236 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f6666-b957-450b-8f4d-602c9ffb375c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:50.048843 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf19a1-bde6-4554-b550-bc71ea31511d None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:50.060820 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-975f6666-b957-450b-8f4d-602c9ffb375c None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:50.062222 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-975f6666-b957-450b-8f4d-602c9ffb375c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:50.068410 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:50.099173 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f05da5-4352-4541-9023-a6efe9fd4dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:50.100545 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f05da5-4352-4541-9023-a6efe9fd4dc5 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:50.101400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f05da5-4352-4541-9023-a6efe9fd4dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:50.102027 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f05da5-4352-4541-9023-a6efe9fd4dc5 None admin] PATH_INFO: `/v3/users/6703d260761041cb8af9777679b5b3ac` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:50.122205 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f05da5-4352-4541-9023-a6efe9fd4dc5 None admin] RBAC: Authorizing `identity:get_user(user_id=6703d260761041cb8af9777679b5b3ac)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:50.124853 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f05da5-4352-4541-9023-a6efe9fd4dc5 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:50.127265 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 123/244] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:37:49 2022] GET /identity/v3/users/6703d260761041cb8af9777679b5b3ac => generated 297 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:37:50.137093 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f05da5-4352-4541-9023-a6efe9fd4dc5 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:50.210469 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 122/245] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 12:37:50.255045 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f6666-b957-450b-8f4d-602c9ffb375c None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:50.286020 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f05da5-4352-4541-9023-a6efe9fd4dc5 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:50.303866 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:50.328284 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ed09a-166a-4e1a-9d27-3f44002d0469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:50.329462 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ed09a-166a-4e1a-9d27-3f44002d0469 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:50.329949 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ed09a-166a-4e1a-9d27-3f44002d0469 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:50.330478 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ed09a-166a-4e1a-9d27-3f44002d0469 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:50.372796 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f6666-b957-450b-8f4d-602c9ffb375c None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:50.377977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:50.408111 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ed09a-166a-4e1a-9d27-3f44002d0469 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:50.410128 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ed09a-166a-4e1a-9d27-3f44002d0469 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:50.412880 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed03a92f-c89d-44d2-950a-9093ba9eb0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:50.414089 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03a92f-c89d-44d2-950a-9093ba9eb0c3 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:50.414477 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03a92f-c89d-44d2-950a-9093ba9eb0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:50.414783 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03a92f-c89d-44d2-950a-9093ba9eb0c3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:50.426893 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed03a92f-c89d-44d2-950a-9093ba9eb0c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:37:50.432512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-ed03a92f-c89d-44d2-950a-9093ba9eb0c3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 12:37:50.432512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:37:50.432512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:37:50.432512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:37:50.432512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:37:50.432512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:37:50.432512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:37:50.432512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:37:50.432512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:37:50.432512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:37:50.432512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:37:50.432512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:37:50.432512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:37:50.432512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:37:50.432512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:37:50.432512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:37:50.433374 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:37:50.433374 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:37:50.433374 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:37:50.433374 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:37:50.433374 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:37:50.433374 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:37:50.433374 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:37:50.433374 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:37:50.433374 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:37:50.433374 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:37:50.433374 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:37:50.433374 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:37:50.433374 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:37:50.433374 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:37:50.433374 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:37:50.433374 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:37:50.434255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:37:50.434255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:37:50.434255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:37:50.434255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:37:50.434255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:37:50.434255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:37:50.434255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:37:50.434255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:37:50.434255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:37:50.434255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:37:50.434255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 12:37:50.434255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:37:50.434255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 123/246] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:37:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:37:50.439580 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed03a92f-c89d-44d2-950a-9093ba9eb0c3 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:50.537317 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 124/247] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:37:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 12:37:50.543787 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ed09a-166a-4e1a-9d27-3f44002d0469 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:50.550018 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:50.553578 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed03a92f-c89d-44d2-950a-9093ba9eb0c3 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:50.593852 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcfb365-5bbc-431c-925d-155fe3cdeb05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:50.594477 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfb365-5bbc-431c-925d-155fe3cdeb05 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:50.594759 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfb365-5bbc-431c-925d-155fe3cdeb05 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:50.595032 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfb365-5bbc-431c-925d-155fe3cdeb05 None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:50.595854 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcfb365-5bbc-431c-925d-155fe3cdeb05 None admin] RBAC: Adding query filter params (name=member) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:50.596157 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcfb365-5bbc-431c-925d-155fe3cdeb05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:50.597737 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcfb365-5bbc-431c-925d-155fe3cdeb05 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:50.607916 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 124/248] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 26 12:37:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:37:50.614791 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcfb365-5bbc-431c-925d-155fe3cdeb05 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:50.635825 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:50.677117 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ed09a-166a-4e1a-9d27-3f44002d0469 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:50.713468 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9918a5-d9f5-45f7-bddb-af742d9d72d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:50.714133 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9918a5-d9f5-45f7-bddb-af742d9d72d4 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:50.714574 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9918a5-d9f5-45f7-bddb-af742d9d72d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:50.714678 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9918a5-d9f5-45f7-bddb-af742d9d72d4 None admin] PATH_INFO: `/v3/projects/eb19926b01264c0a80580d9e18a4397d` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:50.715862 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9918a5-d9f5-45f7-bddb-af742d9d72d4 None admin] RBAC: Authorizing `identity:get_project(project_id=eb19926b01264c0a80580d9e18a4397d)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:50.717324 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9918a5-d9f5-45f7-bddb-af742d9d72d4 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:50.719177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 125/249] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:37:50 2022] GET /identity/v3/projects/eb19926b01264c0a80580d9e18a4397d => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:37:50.724924 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9918a5-d9f5-45f7-bddb-af742d9d72d4 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:50.732864 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcfb365-5bbc-431c-925d-155fe3cdeb05 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:50.773455 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20bb8a-d9fb-4d0e-b790-77779b3b9f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:50.774397 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20bb8a-d9fb-4d0e-b790-77779b3b9f51 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:50.774853 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20bb8a-d9fb-4d0e-b790-77779b3b9f51 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:50.775326 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20bb8a-d9fb-4d0e-b790-77779b3b9f51 None admin] PATH_INFO: `/v3/users/bc90308d618148e49910a19ad2640ab1` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:50.777218 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d20bb8a-d9fb-4d0e-b790-77779b3b9f51 None admin] RBAC: Authorizing `identity:get_user(user_id=bc90308d618148e49910a19ad2640ab1)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:50.778891 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d20bb8a-d9fb-4d0e-b790-77779b3b9f51 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:50.780968 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 125/250] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:37:50 2022] GET /identity/v3/users/bc90308d618148e49910a19ad2640ab1 => generated 297 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:50.786765 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20bb8a-d9fb-4d0e-b790-77779b3b9f51 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:50.824100 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:50.834018 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9918a5-d9f5-45f7-bddb-af742d9d72d4 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:50.874045 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a637413-99ad-42c9-85ec-b9f9737a14cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:50.874666 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a637413-99ad-42c9-85ec-b9f9737a14cb None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:50.874948 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a637413-99ad-42c9-85ec-b9f9737a14cb None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:50.875221 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a637413-99ad-42c9-85ec-b9f9737a14cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:50.876052 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a637413-99ad-42c9-85ec-b9f9737a14cb None admin] RBAC: Adding query filter params (role.id=db16c43424694218852e80f51fb1ee70, scope.project.id=eb19926b01264c0a80580d9e18a4397d, user.id=6703d260761041cb8af9777679b5b3ac) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:50.876370 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a637413-99ad-42c9-85ec-b9f9737a14cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:50.877912 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a637413-99ad-42c9-85ec-b9f9737a14cb None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:50.885962 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 126/251] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:37:50 2022] GET /identity/v3/role_assignments?user.id=6703d260761041cb8af9777679b5b3ac&scope.project.id=eb19926b01264c0a80580d9e18a4397d&role.id=db16c43424694218852e80f51fb1ee70 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:37:50.924421 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20bb8a-d9fb-4d0e-b790-77779b3b9f51 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:50.972572 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c188d1-5851-4716-938f-f2fede694b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:50.973250 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c188d1-5851-4716-938f-f2fede694b56 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:50.973534 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c188d1-5851-4716-938f-f2fede694b56 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:50.973812 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c188d1-5851-4716-938f-f2fede694b56 None admin] PATH_INFO: `/v3/projects/eb19926b01264c0a80580d9e18a4397d` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:50.975202 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c188d1-5851-4716-938f-f2fede694b56 None admin] RBAC: Authorizing `identity:get_project(project_id=eb19926b01264c0a80580d9e18a4397d)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:50.977373 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c188d1-5851-4716-938f-f2fede694b56 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:50.979175 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 126/252] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:37:50 2022] GET /identity/v3/projects/eb19926b01264c0a80580d9e18a4397d => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:37:50.985318 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a637413-99ad-42c9-85ec-b9f9737a14cb None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:50.996800 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:51.125036 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a637413-99ad-42c9-85ec-b9f9737a14cb None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:51.164302 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b64054-9654-49e7-95f7-05243a669c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:51.164986 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b64054-9654-49e7-95f7-05243a669c16 None admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:51.165267 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b64054-9654-49e7-95f7-05243a669c16 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:51.165535 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b64054-9654-49e7-95f7-05243a669c16 None admin] PATH_INFO: `/v3/projects/eb19926b01264c0a80580d9e18a4397d/users/bc90308d618148e49910a19ad2640ab1/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:51.178677 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b64054-9654-49e7-95f7-05243a669c16 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb19926b01264c0a80580d9e18a4397d, user_id=bc90308d618148e49910a19ad2640ab1, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:51.180590 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b64054-9654-49e7-95f7-05243a669c16 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:51.199506 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 127/253] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 26 12:37:50 2022] PUT /identity/v3/projects/eb19926b01264c0a80580d9e18a4397d/users/bc90308d618148e49910a19ad2640ab1/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:37:51.892317 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c460e65-b196-45b3-b660-8b6855b1b6db None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:51.892887 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c460e65-b196-45b3-b660-8b6855b1b6db None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:51.892967 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c460e65-b196-45b3-b660-8b6855b1b6db None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:51.894926 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 127/254] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:37:51.900635 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede502ab-9bff-423d-815c-e8fa2933978e None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:51.900924 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede502ab-9bff-423d-815c-e8fa2933978e None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:51.901207 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede502ab-9bff-423d-815c-e8fa2933978e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:51.928654 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-ede502ab-9bff-423d-815c-e8fa2933978e None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:51.929941 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-ede502ab-9bff-423d-815c-e8fa2933978e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:52.001398 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:52.012200 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 128/255] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:37:52.047861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c460e65-b196-45b3-b660-8b6855b1b6db None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:52.196693 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:52.205301 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c460e65-b196-45b3-b660-8b6855b1b6db None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:52.255989 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27f330e-7982-4516-bcce-c641c2b56449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:52.256644 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f330e-7982-4516-bcce-c641c2b56449 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:52.256957 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f330e-7982-4516-bcce-c641c2b56449 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:52.257803 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f330e-7982-4516-bcce-c641c2b56449 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:52.273175 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27f330e-7982-4516-bcce-c641c2b56449 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:37:52.283184 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-f27f330e-7982-4516-bcce-c641c2b56449 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:37:52.283184 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:37:52.283184 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:37:52.283184 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:37:52.283184 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:37:52.283184 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:37:52.283184 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:37:52.283184 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:37:52.283184 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:37:52.283184 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:37:52.283184 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:37:52.283184 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:37:52.283184 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:37:52.283184 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:37:52.283184 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:37:52.283184 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:37:52.284123 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:37:52.284123 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:37:52.284123 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:37:52.284123 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:37:52.284123 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:37:52.284123 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:37:52.284123 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:37:52.284123 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:37:52.284123 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:37:52.284123 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:37:52.284123 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:37:52.284123 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:37:52.284123 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:37:52.284123 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:37:52.284123 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:37:52.284123 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:37:52.284897 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:37:52.284897 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:37:52.284897 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:37:52.284897 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:37:52.284897 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:37:52.284897 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:37:52.284897 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:37:52.284897 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:37:52.284897 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:37:52.284897 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:37:52.284897 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:37:52.284897 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:37:52.284897 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 128/256] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 26 12:37:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:37:52.292763 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede502ab-9bff-423d-815c-e8fa2933978e None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:52.440231 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede502ab-9bff-423d-815c-e8fa2933978e None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:52.477911 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98aa9f-80ad-4fca-989e-a1d00b9449d2 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:52.478233 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98aa9f-80ad-4fca-989e-a1d00b9449d2 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:52.478497 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98aa9f-80ad-4fca-989e-a1d00b9449d2 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:52.484510 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 129/257] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 12:37:52.491198 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488b50d1-9f74-430e-9a12-1ab58fc1bba9 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:52.491583 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488b50d1-9f74-430e-9a12-1ab58fc1bba9 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:52.491781 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488b50d1-9f74-430e-9a12-1ab58fc1bba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:52.495052 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b58063-1cfa-45e7-adaa-14ce5c8b62ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:52.495708 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b58063-1cfa-45e7-adaa-14ce5c8b62ba None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:52.495994 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b58063-1cfa-45e7-adaa-14ce5c8b62ba None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:52.496269 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b58063-1cfa-45e7-adaa-14ce5c8b62ba None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:52.496999 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b58063-1cfa-45e7-adaa-14ce5c8b62ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:52.497302 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b58063-1cfa-45e7-adaa-14ce5c8b62ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:52.498693 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b58063-1cfa-45e7-adaa-14ce5c8b62ba None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:52.514518 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 129/258] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:37:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:52.520057 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b58063-1cfa-45e7-adaa-14ce5c8b62ba None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:52.529238 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-488b50d1-9f74-430e-9a12-1ab58fc1bba9 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:52.530552 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-488b50d1-9f74-430e-9a12-1ab58fc1bba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:52.652804 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 130/259] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 12:37:52.654207 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:52.700614 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488b50d1-9f74-430e-9a12-1ab58fc1bba9 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:52.739869 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b58063-1cfa-45e7-adaa-14ce5c8b62ba None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:52.830118 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5effbd14-ff88-4a21-baa9-a521bcbf55c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:52.830118 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5effbd14-ff88-4a21-baa9-a521bcbf55c5 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:52.830118 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5effbd14-ff88-4a21-baa9-a521bcbf55c5 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:52.830118 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5effbd14-ff88-4a21-baa9-a521bcbf55c5 None admin] PATH_INFO: `/v3/users/60fc68e2ec8d474387db1dd3b8e0abdf` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:52.831428 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5effbd14-ff88-4a21-baa9-a521bcbf55c5 None admin] RBAC: Authorizing `identity:get_user(user_id=60fc68e2ec8d474387db1dd3b8e0abdf)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:52.833112 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5effbd14-ff88-4a21-baa9-a521bcbf55c5 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:52.836204 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 130/260] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:37:52 2022] GET /identity/v3/users/60fc68e2ec8d474387db1dd3b8e0abdf => generated 283 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:37:52.841354 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5effbd14-ff88-4a21-baa9-a521bcbf55c5 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:52.940380 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:52.990066 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488b50d1-9f74-430e-9a12-1ab58fc1bba9 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:53.001435 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5effbd14-ff88-4a21-baa9-a521bcbf55c5 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:53.049156 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:53.073882 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315db8c8-d375-4f24-90f8-114f35c83421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:53.074522 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315db8c8-d375-4f24-90f8-114f35c83421 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:53.074850 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315db8c8-d375-4f24-90f8-114f35c83421 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:53.075124 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315db8c8-d375-4f24-90f8-114f35c83421 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:53.076980 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab512db-6e7a-42b1-af4e-49e72cd314ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:53.077643 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab512db-6e7a-42b1-af4e-49e72cd314ec None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:53.077929 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab512db-6e7a-42b1-af4e-49e72cd314ec None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:53.078325 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab512db-6e7a-42b1-af4e-49e72cd314ec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:53.116802 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab512db-6e7a-42b1-af4e-49e72cd314ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:37:53.121871 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-aab512db-6e7a-42b1-af4e-49e72cd314ec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 12:37:53.121871 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:37:53.121871 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:37:53.121871 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:37:53.121871 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:37:53.121871 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:37:53.121871 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:37:53.121871 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:37:53.121871 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:37:53.121871 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:37:53.121871 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:37:53.121871 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:37:53.121871 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:37:53.121871 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:37:53.121871 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:37:53.121871 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:37:53.122834 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:37:53.122834 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:37:53.122834 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:37:53.122834 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:37:53.122834 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:37:53.122834 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:37:53.122834 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:37:53.122834 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:37:53.122834 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:37:53.122834 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:37:53.122834 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:37:53.122834 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:37:53.122834 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:37:53.122834 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:37:53.122834 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:37:53.122834 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:37:53.123672 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:37:53.123672 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:37:53.123672 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:37:53.123672 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:37:53.123672 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:37:53.123672 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:37:53.123672 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:37:53.123672 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:37:53.123672 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:37:53.123672 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:37:53.123672 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 12:37:53.123672 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:37:53.123672 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 131/261] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 26 12:37:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 423 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:37:53.131497 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab512db-6e7a-42b1-af4e-49e72cd314ec None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:53.234828 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315db8c8-d375-4f24-90f8-114f35c83421 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:53.236393 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315db8c8-d375-4f24-90f8-114f35c83421 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:53.395539 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab512db-6e7a-42b1-af4e-49e72cd314ec None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:53.452497 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 131/262] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:37:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 12:37:53.460343 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315db8c8-d375-4f24-90f8-114f35c83421 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:53.468038 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3fa5af-fe68-484d-8616-485faa854322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:53.468671 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3fa5af-fe68-484d-8616-485faa854322 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:53.468955 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3fa5af-fe68-484d-8616-485faa854322 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:53.469229 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3fa5af-fe68-484d-8616-485faa854322 None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:53.469966 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3fa5af-fe68-484d-8616-485faa854322 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:53.470273 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3fa5af-fe68-484d-8616-485faa854322 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:53.471702 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3fa5af-fe68-484d-8616-485faa854322 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:53.485391 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 132/263] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:37:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:53.486713 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b685c1a-12a5-43d5-bd0b-c76a5f778416 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:53.487006 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b685c1a-12a5-43d5-bd0b-c76a5f778416 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:53.487285 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b685c1a-12a5-43d5-bd0b-c76a5f778416 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:53.488663 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 133/264] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:37:53.500700 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b685c1a-12a5-43d5-bd0b-c76a5f778416 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:53.506441 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:53.524006 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:53.589563 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315db8c8-d375-4f24-90f8-114f35c83421 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:53.628879 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b666f-d622-4bec-bd23-33b9c199a515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:53.629537 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b666f-d622-4bec-bd23-33b9c199a515 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:53.629823 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b666f-d622-4bec-bd23-33b9c199a515 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:53.630146 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b666f-d622-4bec-bd23-33b9c199a515 None admin] PATH_INFO: `/v3/projects/eb19926b01264c0a80580d9e18a4397d` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:53.631336 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b666f-d622-4bec-bd23-33b9c199a515 None admin] RBAC: Authorizing `identity:get_project(project_id=eb19926b01264c0a80580d9e18a4397d)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:53.633211 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b666f-d622-4bec-bd23-33b9c199a515 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:53.639774 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 132/265] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:37:53 2022] GET /identity/v3/projects/eb19926b01264c0a80580d9e18a4397d => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:37:53.641777 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce482c8-58b5-4222-b457-5020269d1efe None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:53.642075 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce482c8-58b5-4222-b457-5020269d1efe None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:53.642351 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce482c8-58b5-4222-b457-5020269d1efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:53.662625 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-8ce482c8-58b5-4222-b457-5020269d1efe None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:53.664076 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-8ce482c8-58b5-4222-b457-5020269d1efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:53.743166 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 133/266] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:37:53.749157 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce482c8-58b5-4222-b457-5020269d1efe None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:53.778777 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b685c1a-12a5-43d5-bd0b-c76a5f778416 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:53.790894 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:53.849152 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6707aae6-f313-42bd-b709-10ac8aace7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:53.849929 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707aae6-f313-42bd-b709-10ac8aace7d9 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:53.850212 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707aae6-f313-42bd-b709-10ac8aace7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:53.850486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707aae6-f313-42bd-b709-10ac8aace7d9 None admin] PATH_INFO: `/v3/users/60fc68e2ec8d474387db1dd3b8e0abdf` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:53.852429 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6707aae6-f313-42bd-b709-10ac8aace7d9 None admin] RBAC: Authorizing `identity:get_user(user_id=60fc68e2ec8d474387db1dd3b8e0abdf)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:53.853958 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6707aae6-f313-42bd-b709-10ac8aace7d9 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:53.855817 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 134/267] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:37:53 2022] GET /identity/v3/users/60fc68e2ec8d474387db1dd3b8e0abdf => generated 283 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:53.856874 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6707aae6-f313-42bd-b709-10ac8aace7d9 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:53.927474 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce482c8-58b5-4222-b457-5020269d1efe None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:53.948993 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:53.966320 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e978f-3a5d-4649-a7bf-50450f859951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:53.967326 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e978f-3a5d-4649-a7bf-50450f859951 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:53.967864 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e978f-3a5d-4649-a7bf-50450f859951 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:53.968360 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e978f-3a5d-4649-a7bf-50450f859951 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:53.969643 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740e978f-3a5d-4649-a7bf-50450f859951 None admin] RBAC: Adding query filter params (role.id=0e78444e43c744668178971af4087ca0, scope.project.id=eb19926b01264c0a80580d9e18a4397d, user.id=60fc68e2ec8d474387db1dd3b8e0abdf) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:53.969964 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740e978f-3a5d-4649-a7bf-50450f859951 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:53.971854 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740e978f-3a5d-4649-a7bf-50450f859951 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:53.977869 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6707aae6-f313-42bd-b709-10ac8aace7d9 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:53.980237 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 134/268] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:37:53 2022] GET /identity/v3/role_assignments?user.id=60fc68e2ec8d474387db1dd3b8e0abdf&scope.project.id=eb19926b01264c0a80580d9e18a4397d&role.id=0e78444e43c744668178971af4087ca0 => generated 608 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:53.980900 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e978f-3a5d-4649-a7bf-50450f859951 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:54.036273 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf8a21c-e0c1-42a6-a73f-1849e82cf10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:54.036939 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf8a21c-e0c1-42a6-a73f-1849e82cf10c None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:54.037268 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf8a21c-e0c1-42a6-a73f-1849e82cf10c None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:54.037740 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf8a21c-e0c1-42a6-a73f-1849e82cf10c None admin] PATH_INFO: `/v3/domains/default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:54.040840 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf8a21c-e0c1-42a6-a73f-1849e82cf10c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:54.044027 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf8a21c-e0c1-42a6-a73f-1849e82cf10c None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:54.049119 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 135/269] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:37:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:54.053428 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdbd135-2174-4a6c-ad7f-190e958c7978 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:54.053662 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdbd135-2174-4a6c-ad7f-190e958c7978 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:54.053908 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdbd135-2174-4a6c-ad7f-190e958c7978 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:54.055890 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 136/270] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:37:54.061472 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193f64b-3a44-4f41-8bc9-e83bed4d6926 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:54.061757 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193f64b-3a44-4f41-8bc9-e83bed4d6926 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:54.062196 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193f64b-3a44-4f41-8bc9-e83bed4d6926 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:54.063758 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 137/271] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:37:54.064943 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f193f64b-3a44-4f41-8bc9-e83bed4d6926 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:54.103018 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:54.122702 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e978f-3a5d-4649-a7bf-50450f859951 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:54.171393 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c374464c-4bde-44cd-926a-613b667b0dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:54.172290 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374464c-4bde-44cd-926a-613b667b0dc3 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:54.172720 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374464c-4bde-44cd-926a-613b667b0dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:54.173164 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374464c-4bde-44cd-926a-613b667b0dc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:54.197220 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f193f64b-3a44-4f41-8bc9-e83bed4d6926 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:54.243937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885c179f-e206-4c4e-ab3e-95b8811465f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:54.244547 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c179f-e206-4c4e-ab3e-95b8811465f0 None admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:54.245098 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c179f-e206-4c4e-ab3e-95b8811465f0 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:54.245387 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c179f-e206-4c4e-ab3e-95b8811465f0 None admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:54.246420 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885c179f-e206-4c4e-ab3e-95b8811465f0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:54.247926 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885c179f-e206-4c4e-ab3e-95b8811465f0 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:54.274278 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c374464c-4bde-44cd-926a-613b667b0dc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:54.274504 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:54.276746 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c374464c-4bde-44cd-926a-613b667b0dc3 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:54.280703 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 138/272] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Jul 26 12:37:54 2022] POST /identity/v3/users => generated 293 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:37:54.282677 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b83f5-67d2-4524-99d3-6f64809247fa None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:54.283342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b83f5-67d2-4524-99d3-6f64809247fa None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:54.283826 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b83f5-67d2-4524-99d3-6f64809247fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:54.311489 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-1e1b83f5-67d2-4524-99d3-6f64809247fa None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:54.313647 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-1e1b83f5-67d2-4524-99d3-6f64809247fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:54.411072 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:54.419704 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 139/273] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:37:54.419704 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c6970-a9aa-487a-865c-6da7b0293e74 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:54.419704 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c6970-a9aa-487a-865c-6da7b0293e74 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:54.419704 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c6970-a9aa-487a-865c-6da7b0293e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:54.428232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 135/274] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:37:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 12:37:54.430103 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c4035-5cf5-4bdc-bdf5-1db12c6c955b None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:54.430612 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c4035-5cf5-4bdc-bdf5-1db12c6c955b None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:54.431025 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c4035-5cf5-4bdc-bdf5-1db12c6c955b None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:54.432639 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 136/275] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:37:54.438424 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c4035-5cf5-4bdc-bdf5-1db12c6c955b None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:54.444252 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-8b8c6970-a9aa-487a-865c-6da7b0293e74 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:54.449345 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-8b8c6970-a9aa-487a-865c-6da7b0293e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:54.534254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 140/276] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:37:54.535879 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7fc26c-b0e3-4329-976b-70a3bb796e8e None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:54.536230 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7fc26c-b0e3-4329-976b-70a3bb796e8e None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:54.536486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7fc26c-b0e3-4329-976b-70a3bb796e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:54.563101 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-7f7fc26c-b0e3-4329-976b-70a3bb796e8e None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:54.564338 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-7f7fc26c-b0e3-4329-976b-70a3bb796e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:54.573028 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c4035-5cf5-4bdc-bdf5-1db12c6c955b None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:54.620961 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e043de4-fa91-4edd-96b5-5c18687fc70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:54.621615 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e043de4-fa91-4edd-96b5-5c18687fc70b None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:54.621892 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e043de4-fa91-4edd-96b5-5c18687fc70b None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:54.622160 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e043de4-fa91-4edd-96b5-5c18687fc70b None admin] PATH_INFO: `/v3/projects/eb19926b01264c0a80580d9e18a4397d` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:54.623323 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e043de4-fa91-4edd-96b5-5c18687fc70b None admin] RBAC: Authorizing `identity:get_project(project_id=eb19926b01264c0a80580d9e18a4397d)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:54.624859 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e043de4-fa91-4edd-96b5-5c18687fc70b None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:54.626545 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 137/277] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:37:54 2022] GET /identity/v3/projects/eb19926b01264c0a80580d9e18a4397d => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:54.627312 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e043de4-fa91-4edd-96b5-5c18687fc70b None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:54.649053 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:54.669704 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 141/278] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:37:54.670580 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7fc26c-b0e3-4329-976b-70a3bb796e8e None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:54.710120 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:54.773087 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e043de4-fa91-4edd-96b5-5c18687fc70b None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:54.808877 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7fc26c-b0e3-4329-976b-70a3bb796e8e None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:54.817016 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d967a186-dcfe-49fc-b834-ae4d4020d26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:54.817923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d967a186-dcfe-49fc-b834-ae4d4020d26f None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:54.818349 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d967a186-dcfe-49fc-b834-ae4d4020d26f None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:54.818777 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d967a186-dcfe-49fc-b834-ae4d4020d26f None admin] PATH_INFO: `/v3/roles/member` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:54.832179 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d967a186-dcfe-49fc-b834-ae4d4020d26f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:37:54.837553 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-d967a186-dcfe-49fc-b834-ae4d4020d26f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 12:37:54.837553 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:37:54.837553 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:37:54.837553 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:37:54.837553 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:37:54.837553 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:37:54.837553 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:37:54.837553 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:37:54.837553 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:37:54.837553 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:37:54.837553 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:37:54.837553 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:37:54.837553 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:37:54.837553 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:37:54.837553 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:37:54.837553 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:37:54.838712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:37:54.838712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:37:54.838712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:37:54.838712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:37:54.838712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:37:54.838712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:37:54.838712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:37:54.838712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:37:54.838712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:37:54.838712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:37:54.838712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:37:54.838712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:37:54.838712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:37:54.838712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:37:54.838712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:37:54.838712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:37:54.839671 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:37:54.839671 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:37:54.839671 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:37:54.839671 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:37:54.839671 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:37:54.839671 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:37:54.839671 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:37:54.839671 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:37:54.839671 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:37:54.839671 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:37:54.839671 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 12:37:54.839671 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:37:54.841586 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 138/279] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:37:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:37:54.847246 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d967a186-dcfe-49fc-b834-ae4d4020d26f None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:54.851399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eaea80-e81d-4e90-b61b-4b77114ad0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:54.852084 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eaea80-e81d-4e90-b61b-4b77114ad0a4 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:54.852361 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eaea80-e81d-4e90-b61b-4b77114ad0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:54.852631 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eaea80-e81d-4e90-b61b-4b77114ad0a4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:54.867716 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eaea80-e81d-4e90-b61b-4b77114ad0a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:37:54.873447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-67eaea80-e81d-4e90-b61b-4b77114ad0a4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 12:37:54.873447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:37:54.873447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:37:54.873447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:37:54.873447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:37:54.873447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:37:54.873447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:37:54.873447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:37:54.873447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:37:54.873447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:37:54.873447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:37:54.873447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:37:54.873447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:37:54.873447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:37:54.873447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:37:54.873447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:37:54.874349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:37:54.874349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:37:54.874349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:37:54.874349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:37:54.874349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:37:54.874349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:37:54.874349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:37:54.874349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:37:54.874349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:37:54.874349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:37:54.874349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:37:54.874349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:37:54.874349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:37:54.874349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:37:54.874349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:37:54.874349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:37:54.876374 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:37:54.876374 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:37:54.876374 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:37:54.876374 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:37:54.876374 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:37:54.876374 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:37:54.876374 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:37:54.876374 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:37:54.876374 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:37:54.876374 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:37:54.876374 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 12:37:54.876374 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:37:54.878903 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 142/280] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:37:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:37:54.880074 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eaea80-e81d-4e90-b61b-4b77114ad0a4 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:54.946233 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:54.983991 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d967a186-dcfe-49fc-b834-ae4d4020d26f None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:54.999994 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:55.020564 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eaea80-e81d-4e90-b61b-4b77114ad0a4 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:55.022358 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0420dbe-fbcb-4714-8f63-dbadf4157350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:55.023244 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0420dbe-fbcb-4714-8f63-dbadf4157350 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:55.023767 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0420dbe-fbcb-4714-8f63-dbadf4157350 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:55.024240 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0420dbe-fbcb-4714-8f63-dbadf4157350 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:55.025222 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0420dbe-fbcb-4714-8f63-dbadf4157350 None admin] RBAC: Adding query filter params (role.id=6589779ddd834447a1ed1921e1c834f6, scope.project.id=eb19926b01264c0a80580d9e18a4397d, user.id=60fc68e2ec8d474387db1dd3b8e0abdf) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:55.025745 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0420dbe-fbcb-4714-8f63-dbadf4157350 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:55.027392 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0420dbe-fbcb-4714-8f63-dbadf4157350 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:55.033971 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 139/281] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:37:54 2022] GET /identity/v3/role_assignments?user.id=60fc68e2ec8d474387db1dd3b8e0abdf&scope.project.id=eb19926b01264c0a80580d9e18a4397d&role.id=6589779ddd834447a1ed1921e1c834f6 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:55.035151 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0420dbe-fbcb-4714-8f63-dbadf4157350 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:55.100484 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a18dc59-4bab-4f3a-90c9-8198fb825a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:55.101797 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18dc59-4bab-4f3a-90c9-8198fb825a17 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:55.102080 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18dc59-4bab-4f3a-90c9-8198fb825a17 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:55.102354 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18dc59-4bab-4f3a-90c9-8198fb825a17 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:55.140333 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a18dc59-4bab-4f3a-90c9-8198fb825a17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:37:55.145399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-3a18dc59-4bab-4f3a-90c9-8198fb825a17 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 12:37:55.145399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:37:55.145399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:37:55.145399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:37:55.145399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:37:55.145399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:37:55.145399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:37:55.145399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:37:55.145399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:37:55.145399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:37:55.145399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:37:55.145399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:37:55.145399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:37:55.145399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:37:55.145399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:37:55.145399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:37:55.146251 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:37:55.146251 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:37:55.146251 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:37:55.146251 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:37:55.146251 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:37:55.146251 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:37:55.146251 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:37:55.146251 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:37:55.146251 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:37:55.146251 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:37:55.146251 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:37:55.146251 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:37:55.146251 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:37:55.146251 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:37:55.146251 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:37:55.146251 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:37:55.147462 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:37:55.147462 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:37:55.147462 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:37:55.147462 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:37:55.147462 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:37:55.147462 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:37:55.147462 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:37:55.147462 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:37:55.147462 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:37:55.147462 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:37:55.147462 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 12:37:55.147462 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:37:55.147462 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 143/282] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:37:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:37:55.148772 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a18dc59-4bab-4f3a-90c9-8198fb825a17 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:55.164376 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0420dbe-fbcb-4714-8f63-dbadf4157350 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:55.208147 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fae00e-a239-445b-b3aa-34586b611458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:55.209296 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fae00e-a239-445b-b3aa-34586b611458 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:55.209778 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fae00e-a239-445b-b3aa-34586b611458 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:55.210888 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fae00e-a239-445b-b3aa-34586b611458 None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:55.212043 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fae00e-a239-445b-b3aa-34586b611458 None admin] RBAC: Adding query filter params (name=member) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:55.212573 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fae00e-a239-445b-b3aa-34586b611458 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:55.214495 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fae00e-a239-445b-b3aa-34586b611458 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:55.226041 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 140/283] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 26 12:37:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:55.227525 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fae00e-a239-445b-b3aa-34586b611458 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:55.281006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:55.358142 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fae00e-a239-445b-b3aa-34586b611458 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:55.368193 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a18dc59-4bab-4f3a-90c9-8198fb825a17 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:55.382300 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:55.411953 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8695b5-a758-4202-b2a5-5223c042c8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:55.412613 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8695b5-a758-4202-b2a5-5223c042c8df None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:55.412899 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8695b5-a758-4202-b2a5-5223c042c8df None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:55.413214 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8695b5-a758-4202-b2a5-5223c042c8df None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:55.414468 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8695b5-a758-4202-b2a5-5223c042c8df None admin] RBAC: Adding query filter params (name=member) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:55.414778 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8695b5-a758-4202-b2a5-5223c042c8df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:55.416952 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8695b5-a758-4202-b2a5-5223c042c8df None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:55.425134 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9773755-349c-46d9-a197-a130cdeb660f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:55.426397 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9773755-349c-46d9-a197-a130cdeb660f None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:55.426891 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9773755-349c-46d9-a197-a130cdeb660f None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:55.427364 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9773755-349c-46d9-a197-a130cdeb660f None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:55.428221 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 141/284] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 26 12:37:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:55.428696 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9773755-349c-46d9-a197-a130cdeb660f None admin] RBAC: Adding query filter params (name=reader) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:55.429118 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8695b5-a758-4202-b2a5-5223c042c8df None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:55.429556 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9773755-349c-46d9-a197-a130cdeb660f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:55.431234 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9773755-349c-46d9-a197-a130cdeb660f None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:55.448195 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 144/285] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 26 12:37:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:55.448766 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9773755-349c-46d9-a197-a130cdeb660f None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:55.553153 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8695b5-a758-4202-b2a5-5223c042c8df None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:55.555817 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:55.561232 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9773755-349c-46d9-a197-a130cdeb660f None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:55.594635 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca8fe72-d2a0-46bd-b3de-487b2fcab797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:55.595716 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8fe72-d2a0-46bd-b3de-487b2fcab797 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:55.596277 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8fe72-d2a0-46bd-b3de-487b2fcab797 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:55.596753 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8fe72-d2a0-46bd-b3de-487b2fcab797 None admin] PATH_INFO: `/v3/users/b0b1eec2f7864118bb6144c335592d15` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:55.599987 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58fd1b-22ce-4836-ab91-1dad52ad22b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:55.601800 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58fd1b-22ce-4836-ab91-1dad52ad22b6 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:55.601800 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58fd1b-22ce-4836-ab91-1dad52ad22b6 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:55.601800 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58fd1b-22ce-4836-ab91-1dad52ad22b6 None admin] PATH_INFO: `/v3/users/bc90308d618148e49910a19ad2640ab1` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:55.602960 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf58fd1b-22ce-4836-ab91-1dad52ad22b6 None admin] RBAC: Authorizing `identity:get_user(user_id=bc90308d618148e49910a19ad2640ab1)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:55.604464 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf58fd1b-22ce-4836-ab91-1dad52ad22b6 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:55.606109 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 145/286] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:37:55 2022] GET /identity/v3/users/bc90308d618148e49910a19ad2640ab1 => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:55.612398 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58fd1b-22ce-4836-ab91-1dad52ad22b6 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:55.618707 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca8fe72-d2a0-46bd-b3de-487b2fcab797 None admin] RBAC: Authorizing `identity:get_user(user_id=b0b1eec2f7864118bb6144c335592d15)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:55.620577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca8fe72-d2a0-46bd-b3de-487b2fcab797 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:55.622751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 142/287] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:37:55 2022] GET /identity/v3/users/b0b1eec2f7864118bb6144c335592d15 => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:55.623948 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca8fe72-d2a0-46bd-b3de-487b2fcab797 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:55.664118 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:55.728462 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58fd1b-22ce-4836-ab91-1dad52ad22b6 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:55.742327 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca8fe72-d2a0-46bd-b3de-487b2fcab797 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:55.768483 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf052ae-6886-4970-9bef-aa5863839500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:55.769148 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf052ae-6886-4970-9bef-aa5863839500 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:55.769416 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf052ae-6886-4970-9bef-aa5863839500 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:55.769698 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf052ae-6886-4970-9bef-aa5863839500 None admin] PATH_INFO: `/v3/users/6703d260761041cb8af9777679b5b3ac` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:55.770928 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf052ae-6886-4970-9bef-aa5863839500 None admin] RBAC: Authorizing `identity:get_user(user_id=6703d260761041cb8af9777679b5b3ac)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:55.772497 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf052ae-6886-4970-9bef-aa5863839500 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:55.774870 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 146/288] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:37:55 2022] GET /identity/v3/users/6703d260761041cb8af9777679b5b3ac => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:55.781372 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf052ae-6886-4970-9bef-aa5863839500 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:55.806552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d3ccd-e6a5-4aba-a3e3-9d005bf1bd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:55.807312 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d3ccd-e6a5-4aba-a3e3-9d005bf1bd6f None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:55.807466 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d3ccd-e6a5-4aba-a3e3-9d005bf1bd6f None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:55.807779 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d3ccd-e6a5-4aba-a3e3-9d005bf1bd6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:55.839771 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:55.888108 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84d3ccd-e6a5-4aba-a3e3-9d005bf1bd6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:55.889752 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84d3ccd-e6a5-4aba-a3e3-9d005bf1bd6f None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:55.923326 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf052ae-6886-4970-9bef-aa5863839500 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:55.935107 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:55.963497 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d718cf9-b442-4abe-a832-2ca6e87dec9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:55.964223 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d718cf9-b442-4abe-a832-2ca6e87dec9e None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:55.964432 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d718cf9-b442-4abe-a832-2ca6e87dec9e None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:55.964707 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d718cf9-b442-4abe-a832-2ca6e87dec9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:55.965521 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d718cf9-b442-4abe-a832-2ca6e87dec9e None admin] RBAC: Adding query filter params (role.id=01c9179cca2747a6b95b0decfca8b7ae, user.id=b0b1eec2f7864118bb6144c335592d15, scope.system=all) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:55.965971 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d718cf9-b442-4abe-a832-2ca6e87dec9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:55.967367 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d718cf9-b442-4abe-a832-2ca6e87dec9e None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:55.972802 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 147/289] 10.222.0.20 () {56 vars in 1308 bytes} [Tue Jul 26 12:37:55 2022] GET /identity/v3/role_assignments?user.id=b0b1eec2f7864118bb6144c335592d15&scope.system=all&role.id=01c9179cca2747a6b95b0decfca8b7ae => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:55.973395 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d718cf9-b442-4abe-a832-2ca6e87dec9e None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:56.018353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 143/290] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:37:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 12:37:56.025062 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d3ccd-e6a5-4aba-a3e3-9d005bf1bd6f None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:56.165725 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:56.203913 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d718cf9-b442-4abe-a832-2ca6e87dec9e None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:56.228029 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d3ccd-e6a5-4aba-a3e3-9d005bf1bd6f None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:56.244042 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7705033-99f2-4cbf-9d56-daaee86a1630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:56.244794 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7705033-99f2-4cbf-9d56-daaee86a1630 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:56.245186 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7705033-99f2-4cbf-9d56-daaee86a1630 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:56.245448 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7705033-99f2-4cbf-9d56-daaee86a1630 None admin] PATH_INFO: `/v3/projects/eb19926b01264c0a80580d9e18a4397d` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:56.247753 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7705033-99f2-4cbf-9d56-daaee86a1630 None admin] RBAC: Authorizing `identity:get_project(project_id=eb19926b01264c0a80580d9e18a4397d)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:56.249414 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7705033-99f2-4cbf-9d56-daaee86a1630 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:56.251358 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 148/291] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:37:55 2022] GET /identity/v3/projects/eb19926b01264c0a80580d9e18a4397d => generated 302 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:56.256860 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7705033-99f2-4cbf-9d56-daaee86a1630 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:56.274453 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b54fb68-b9fb-4e62-8aae-db4c7ffa9541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:56.275111 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54fb68-b9fb-4e62-8aae-db4c7ffa9541 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:56.275388 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54fb68-b9fb-4e62-8aae-db4c7ffa9541 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:56.275688 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54fb68-b9fb-4e62-8aae-db4c7ffa9541 None admin] PATH_INFO: `/v3/projects/eb19926b01264c0a80580d9e18a4397d` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:56.280894 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b54fb68-b9fb-4e62-8aae-db4c7ffa9541 None admin] RBAC: Authorizing `identity:get_project(project_id=eb19926b01264c0a80580d9e18a4397d)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:56.282357 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b54fb68-b9fb-4e62-8aae-db4c7ffa9541 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:56.284269 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 144/292] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:37:56 2022] GET /identity/v3/projects/eb19926b01264c0a80580d9e18a4397d => generated 302 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:56.292349 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b54fb68-b9fb-4e62-8aae-db4c7ffa9541 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:56.324696 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:56.388880 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7705033-99f2-4cbf-9d56-daaee86a1630 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:56.429571 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa902d-cc1e-4257-abd4-34593b9adc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:56.430168 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa902d-cc1e-4257-abd4-34593b9adc53 None admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:56.430449 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa902d-cc1e-4257-abd4-34593b9adc53 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:56.430727 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa902d-cc1e-4257-abd4-34593b9adc53 None admin] PATH_INFO: `/v3/projects/eb19926b01264c0a80580d9e18a4397d/users/6703d260761041cb8af9777679b5b3ac/roles/db16c43424694218852e80f51fb1ee70` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:56.432804 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa902d-cc1e-4257-abd4-34593b9adc53 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb19926b01264c0a80580d9e18a4397d, user_id=6703d260761041cb8af9777679b5b3ac, role_id=db16c43424694218852e80f51fb1ee70)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:56.434390 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa902d-cc1e-4257-abd4-34593b9adc53 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:56.452626 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 149/293] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 26 12:37:56 2022] PUT /identity/v3/projects/eb19926b01264c0a80580d9e18a4397d/users/6703d260761041cb8af9777679b5b3ac/roles/db16c43424694218852e80f51fb1ee70 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:37:56.494011 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b54fb68-b9fb-4e62-8aae-db4c7ffa9541 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:56.538921 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:56.548948 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12423924-cec3-4c10-a3ba-ced9e74ea031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:56.549574 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12423924-cec3-4c10-a3ba-ced9e74ea031 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:56.549852 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12423924-cec3-4c10-a3ba-ced9e74ea031 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:56.550118 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12423924-cec3-4c10-a3ba-ced9e74ea031 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:56.550912 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12423924-cec3-4c10-a3ba-ced9e74ea031 None admin] RBAC: Adding query filter params (role.id=01c9179cca2747a6b95b0decfca8b7ae, scope.project.id=eb19926b01264c0a80580d9e18a4397d, user.id=bc90308d618148e49910a19ad2640ab1) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:56.551221 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12423924-cec3-4c10-a3ba-ced9e74ea031 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:56.552718 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12423924-cec3-4c10-a3ba-ced9e74ea031 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:56.562073 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 145/294] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:37:56 2022] GET /identity/v3/role_assignments?user.id=bc90308d618148e49910a19ad2640ab1&scope.project.id=eb19926b01264c0a80580d9e18a4397d&role.id=01c9179cca2747a6b95b0decfca8b7ae => generated 608 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:56.675389 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84742136-754b-43bf-bd0b-99b439607b7f None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:56.675798 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84742136-754b-43bf-bd0b-99b439607b7f None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:56.675989 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84742136-754b-43bf-bd0b-99b439607b7f None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:56.677901 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 150/295] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 12:37:56.684035 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd17d45-ad9e-49d7-ba53-cf641f431936 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:56.684298 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd17d45-ad9e-49d7-ba53-cf641f431936 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:56.684515 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd17d45-ad9e-49d7-ba53-cf641f431936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:56.712860 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-ccd17d45-ad9e-49d7-ba53-cf641f431936 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:56.714152 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-ccd17d45-ad9e-49d7-ba53-cf641f431936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:56.785821 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434181cc-f5a1-40e0-931f-c14726691c43 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:56.786132 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434181cc-f5a1-40e0-931f-c14726691c43 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:56.786361 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434181cc-f5a1-40e0-931f-c14726691c43 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:56.787953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 151/296] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 12:37:56.793917 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8629db0-cf16-4258-a760-0aaa31a3850b None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:56.794206 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8629db0-cf16-4258-a760-0aaa31a3850b None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:56.794478 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8629db0-cf16-4258-a760-0aaa31a3850b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:56.811021 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 146/297] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:37:56.823010 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-f8629db0-cf16-4258-a760-0aaa31a3850b None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:56.824248 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-f8629db0-cf16-4258-a760-0aaa31a3850b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:56.839097 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd17d45-ad9e-49d7-ba53-cf641f431936 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:56.902482 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 152/298] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 12:37:56.926750 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8629db0-cf16-4258-a760-0aaa31a3850b None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:56.958220 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd17d45-ad9e-49d7-ba53-cf641f431936 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:56.963092 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:56.991461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:57.009653 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6777900-4733-482a-b63f-766c8e77e437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:57.010385 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6777900-4733-482a-b63f-766c8e77e437 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:57.010771 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6777900-4733-482a-b63f-766c8e77e437 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:57.011086 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6777900-4733-482a-b63f-766c8e77e437 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:57.026721 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6777900-4733-482a-b63f-766c8e77e437 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:37:57.032402 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-c6777900-4733-482a-b63f-766c8e77e437 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 12:37:57.032402 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:37:57.032402 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:37:57.032402 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:37:57.032402 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:37:57.032402 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:37:57.032402 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:37:57.032402 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:37:57.032402 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:37:57.032402 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:37:57.032402 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:37:57.032402 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:37:57.032402 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:37:57.032402 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:37:57.032402 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:37:57.032402 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:37:57.034353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:37:57.034353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:37:57.034353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:37:57.034353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:37:57.034353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:37:57.034353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:37:57.034353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:37:57.034353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:37:57.034353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:37:57.034353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:37:57.034353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:37:57.034353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:37:57.034353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:37:57.034353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:37:57.034353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:37:57.034353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:37:57.036328 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:37:57.036328 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:37:57.036328 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:37:57.036328 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:37:57.036328 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:37:57.036328 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:37:57.036328 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:37:57.036328 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:37:57.036328 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:37:57.036328 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:37:57.036328 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 12:37:57.036328 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:37:57.036328 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 147/299] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:37:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:37:57.041666 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6777900-4733-482a-b63f-766c8e77e437 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:57.072433 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8629db0-cf16-4258-a760-0aaa31a3850b None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:57.123018 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8dbf74-ef0e-463c-8d6d-818db43f30cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:57.123852 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8dbf74-ef0e-463c-8d6d-818db43f30cb None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:57.124136 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8dbf74-ef0e-463c-8d6d-818db43f30cb None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:57.124413 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8dbf74-ef0e-463c-8d6d-818db43f30cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:57.125544 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8dbf74-ef0e-463c-8d6d-818db43f30cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:57.127001 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8dbf74-ef0e-463c-8d6d-818db43f30cb None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:57.128995 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 153/300] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:37:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:37:57.134990 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8dbf74-ef0e-463c-8d6d-818db43f30cb None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:57.166637 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6777900-4733-482a-b63f-766c8e77e437 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:57.210187 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd5e3c-26c8-499e-b980-af54febebedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:57.211094 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd5e3c-26c8-499e-b980-af54febebedb None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:57.211584 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd5e3c-26c8-499e-b980-af54febebedb None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:57.212151 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd5e3c-26c8-499e-b980-af54febebedb None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:57.213237 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbd5e3c-26c8-499e-b980-af54febebedb None admin] RBAC: Adding query filter params (name=reader) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:57.213882 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbd5e3c-26c8-499e-b980-af54febebedb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:57.215466 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbd5e3c-26c8-499e-b980-af54febebedb None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:57.227193 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 148/301] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 26 12:37:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:37:57.232334 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd5e3c-26c8-499e-b980-af54febebedb None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:57.273565 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:57.280708 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8dbf74-ef0e-463c-8d6d-818db43f30cb None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:57.282801 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:57.329770 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcdfba4-89ee-4b0d-abd9-d38fdb261caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:57.330393 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcdfba4-89ee-4b0d-abd9-d38fdb261caa None admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:57.330675 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcdfba4-89ee-4b0d-abd9-d38fdb261caa None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:57.330947 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcdfba4-89ee-4b0d-abd9-d38fdb261caa None admin] PATH_INFO: `/v3/groups` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:57.331613 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcdfba4-89ee-4b0d-abd9-d38fdb261caa None admin] RBAC: Authorizing `identity:create_group()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:57.333084 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcdfba4-89ee-4b0d-abd9-d38fdb261caa None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:57.359665 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 154/302] 10.222.0.20 () {60 vars in 1134 bytes} [Tue Jul 26 12:37:57 2022] POST /identity/v3/groups => generated 229 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:37:57.361528 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd5e3c-26c8-499e-b980-af54febebedb None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:57.403540 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58c92a4-350c-4c06-b0ea-7cdf7ecc673a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:57.404268 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c92a4-350c-4c06-b0ea-7cdf7ecc673a None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:57.404549 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c92a4-350c-4c06-b0ea-7cdf7ecc673a None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:57.404876 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c92a4-350c-4c06-b0ea-7cdf7ecc673a None admin] PATH_INFO: `/v3/users/5d01e35f85964c59a9ea5bb44bef9dbb` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:57.421997 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58c92a4-350c-4c06-b0ea-7cdf7ecc673a None admin] RBAC: Authorizing `identity:get_user(user_id=5d01e35f85964c59a9ea5bb44bef9dbb)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:57.423562 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58c92a4-350c-4c06-b0ea-7cdf7ecc673a None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:57.425901 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 149/303] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:37:57 2022] GET /identity/v3/users/5d01e35f85964c59a9ea5bb44bef9dbb => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:37:57.432826 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcdfba4-89ee-4b0d-abd9-d38fdb261caa None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:57.550884 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcdfba4-89ee-4b0d-abd9-d38fdb261caa None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:57.586618 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6181a86d-ccf0-455f-a65e-7f713399a366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:57.587564 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181a86d-ccf0-455f-a65e-7f713399a366 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:57.587997 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181a86d-ccf0-455f-a65e-7f713399a366 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:57.588407 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181a86d-ccf0-455f-a65e-7f713399a366 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:57.589333 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6181a86d-ccf0-455f-a65e-7f713399a366 None admin] RBAC: Adding query filter params (role.id=db16c43424694218852e80f51fb1ee70, user.id=5d01e35f85964c59a9ea5bb44bef9dbb, scope.system=all) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:57.589797 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6181a86d-ccf0-455f-a65e-7f713399a366 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:57.591414 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6181a86d-ccf0-455f-a65e-7f713399a366 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:57.597197 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 155/304] 10.222.0.20 () {56 vars in 1308 bytes} [Tue Jul 26 12:37:57 2022] GET /identity/v3/role_assignments?user.id=5d01e35f85964c59a9ea5bb44bef9dbb&scope.system=all&role.id=db16c43424694218852e80f51fb1ee70 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:58.238512 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554add9-c854-4b08-bedc-2d08cf00a460 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:58.239802 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554add9-c854-4b08-bedc-2d08cf00a460 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:58.240272 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554add9-c854-4b08-bedc-2d08cf00a460 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:58.241861 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 150/305] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:37:58.247314 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c48fa6-3b08-48a9-abb4-268295137164 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:58.247858 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c48fa6-3b08-48a9-abb4-268295137164 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:58.248327 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c48fa6-3b08-48a9-abb4-268295137164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:58.266524 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-31c48fa6-3b08-48a9-abb4-268295137164 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:58.268095 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-31c48fa6-3b08-48a9-abb4-268295137164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:58.287755 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:58.343832 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 156/306] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:37:58.365362 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3554add9-c854-4b08-bedc-2d08cf00a460 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:58.471589 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3554add9-c854-4b08-bedc-2d08cf00a460 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:58.473094 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:58.510432 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189618c3-b92c-47c6-90fc-7c4e4ce0dca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:58.511071 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189618c3-b92c-47c6-90fc-7c4e4ce0dca5 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:58.511346 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189618c3-b92c-47c6-90fc-7c4e4ce0dca5 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:58.511614 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189618c3-b92c-47c6-90fc-7c4e4ce0dca5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:58.522160 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189618c3-b92c-47c6-90fc-7c4e4ce0dca5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:37:58.527344 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-189618c3-b92c-47c6-90fc-7c4e4ce0dca5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 12:37:58.527344 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:37:58.527344 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:37:58.527344 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:37:58.527344 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:37:58.527344 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:37:58.527344 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:37:58.527344 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:37:58.527344 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:37:58.527344 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:37:58.527344 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:37:58.527344 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:37:58.527344 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:37:58.527344 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:37:58.527344 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:37:58.527344 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:37:58.528240 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:37:58.528240 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:37:58.528240 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:37:58.528240 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:37:58.528240 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:37:58.528240 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:37:58.528240 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:37:58.528240 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:37:58.528240 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:37:58.528240 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:37:58.528240 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:37:58.528240 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:37:58.528240 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:37:58.528240 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:37:58.528240 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:37:58.528240 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:37:58.529020 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:37:58.529020 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:37:58.529020 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:37:58.529020 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:37:58.529020 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:37:58.529020 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:37:58.529020 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:37:58.529020 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:37:58.529020 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:37:58.529020 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:37:58.529020 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 12:37:58.529020 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:37:58.533310 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 151/307] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:37:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:37:58.534673 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c48fa6-3b08-48a9-abb4-268295137164 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:58.650035 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c48fa6-3b08-48a9-abb4-268295137164 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:58.692220 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e323da-2fcf-4e3a-96e1-04595eef95bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:58.693216 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e323da-2fcf-4e3a-96e1-04595eef95bd None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:58.693696 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e323da-2fcf-4e3a-96e1-04595eef95bd None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:58.694189 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e323da-2fcf-4e3a-96e1-04595eef95bd None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:58.695131 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e323da-2fcf-4e3a-96e1-04595eef95bd None admin] RBAC: Adding query filter params (name=member) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:58.695630 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e323da-2fcf-4e3a-96e1-04595eef95bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:58.697220 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e323da-2fcf-4e3a-96e1-04595eef95bd None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:58.706424 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 157/308] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 26 12:37:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:58.711981 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189618c3-b92c-47c6-90fc-7c4e4ce0dca5 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:58.795439 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25233291-6e6c-4c51-bd9a-27ddd921816f None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:58.795757 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25233291-6e6c-4c51-bd9a-27ddd921816f None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:58.795952 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25233291-6e6c-4c51-bd9a-27ddd921816f None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:58.797557 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 158/309] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:37:58.802349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e6491-fde9-434a-a5fc-cd261690a32e None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:58.802634 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e6491-fde9-434a-a5fc-cd261690a32e None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:58.802913 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e6491-fde9-434a-a5fc-cd261690a32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:58.821128 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-084e6491-fde9-434a-a5fc-cd261690a32e None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:58.822321 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-084e6491-fde9-434a-a5fc-cd261690a32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:58.829231 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189618c3-b92c-47c6-90fc-7c4e4ce0dca5 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:58.831983 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:58.868938 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b4379-3469-42b2-a566-7ec91d8230d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:58.869751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b4379-3469-42b2-a566-7ec91d8230d1 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:58.870035 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b4379-3469-42b2-a566-7ec91d8230d1 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:58.870306 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b4379-3469-42b2-a566-7ec91d8230d1 None admin] PATH_INFO: `/v3/users/b0b1eec2f7864118bb6144c335592d15` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:58.871693 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250b4379-3469-42b2-a566-7ec91d8230d1 None admin] RBAC: Authorizing `identity:get_user(user_id=b0b1eec2f7864118bb6144c335592d15)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:58.873146 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250b4379-3469-42b2-a566-7ec91d8230d1 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:58.874748 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 152/310] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:37:58 2022] GET /identity/v3/users/b0b1eec2f7864118bb6144c335592d15 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:58.880707 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b4379-3469-42b2-a566-7ec91d8230d1 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:58.916131 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:58.917908 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 159/311] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 12:37:58.941641 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084e6491-fde9-434a-a5fc-cd261690a32e None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:59.003309 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b4379-3469-42b2-a566-7ec91d8230d1 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:59.041626 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d76f8d9-ede0-4eff-81b5-7acff646884d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:59.042797 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76f8d9-ede0-4eff-81b5-7acff646884d None admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:59.043303 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76f8d9-ede0-4eff-81b5-7acff646884d None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:59.043808 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76f8d9-ede0-4eff-81b5-7acff646884d None admin] PATH_INFO: `/v3/system/users/b0b1eec2f7864118bb6144c335592d15/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:59.044761 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084e6491-fde9-434a-a5fc-cd261690a32e None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:59.045708 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d76f8d9-ede0-4eff-81b5-7acff646884d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b0b1eec2f7864118bb6144c335592d15, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:59.047513 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d76f8d9-ede0-4eff-81b5-7acff646884d None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:59.059831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 153/312] 10.222.0.20 () {58 vars in 1334 bytes} [Tue Jul 26 12:37:58 2022] PUT /identity/v3/system/users/b0b1eec2f7864118bb6144c335592d15/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:37:59.090439 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f2afa5-31b3-4790-b762-fb47abff5b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:59.091112 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2afa5-31b3-4790-b762-fb47abff5b90 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:59.091393 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2afa5-31b3-4790-b762-fb47abff5b90 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:59.091676 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2afa5-31b3-4790-b762-fb47abff5b90 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:59.105913 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f2afa5-31b3-4790-b762-fb47abff5b90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:37:59.111385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-c8f2afa5-31b3-4790-b762-fb47abff5b90 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 12:37:59.111385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:37:59.111385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:37:59.111385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:37:59.111385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:37:59.111385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:37:59.111385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:37:59.111385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:37:59.111385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:37:59.111385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:37:59.111385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:37:59.111385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:37:59.111385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:37:59.111385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:37:59.111385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:37:59.111385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:37:59.112379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:37:59.112379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:37:59.112379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:37:59.112379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:37:59.112379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:37:59.112379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:37:59.112379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:37:59.112379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:37:59.112379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:37:59.112379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:37:59.112379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:37:59.112379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:37:59.112379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:37:59.112379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:37:59.112379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:37:59.112379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:37:59.113173 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:37:59.113173 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:37:59.113173 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:37:59.113173 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:37:59.113173 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:37:59.113173 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:37:59.113173 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:37:59.113173 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:37:59.113173 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:37:59.113173 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:37:59.113173 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 12:37:59.113173 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:37:59.113173 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 160/313] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:37:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:37:59.120091 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d76f8d9-ede0-4eff-81b5-7acff646884d None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:59.248074 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:59.254449 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d76f8d9-ede0-4eff-81b5-7acff646884d None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:59.291324 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9456ed9-e0ec-4004-9140-bb486811aa9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:59.292219 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9456ed9-e0ec-4004-9140-bb486811aa9e None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:59.292639 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9456ed9-e0ec-4004-9140-bb486811aa9e None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:59.293084 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9456ed9-e0ec-4004-9140-bb486811aa9e None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:59.293952 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9456ed9-e0ec-4004-9140-bb486811aa9e None admin] RBAC: Adding query filter params (name=reader) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:37:59.294397 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9456ed9-e0ec-4004-9140-bb486811aa9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:59.296000 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9456ed9-e0ec-4004-9140-bb486811aa9e None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:59.305936 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 154/314] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 26 12:37:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:59.311241 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f2afa5-31b3-4790-b762-fb47abff5b90 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:59.315799 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:59.422754 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f2afa5-31b3-4790-b762-fb47abff5b90 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:59.459679 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3224dfcc-38ef-4f01-acb6-ecefa809240f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:59.460766 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224dfcc-38ef-4f01-acb6-ecefa809240f None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:59.461305 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224dfcc-38ef-4f01-acb6-ecefa809240f None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:59.461778 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224dfcc-38ef-4f01-acb6-ecefa809240f None admin] PATH_INFO: `/v3/users/6703d260761041cb8af9777679b5b3ac` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:59.463152 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3224dfcc-38ef-4f01-acb6-ecefa809240f None admin] RBAC: Authorizing `identity:get_user(user_id=6703d260761041cb8af9777679b5b3ac)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:59.464844 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3224dfcc-38ef-4f01-acb6-ecefa809240f None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:59.466957 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 161/315] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:37:59 2022] GET /identity/v3/users/6703d260761041cb8af9777679b5b3ac => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:37:59.472641 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9456ed9-e0ec-4004-9140-bb486811aa9e None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:59.520622 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b0927-ebd9-4e60-b4de-5f6816d8f9de None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:59.521265 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b0927-ebd9-4e60-b4de-5f6816d8f9de None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:59.522175 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b0927-ebd9-4e60-b4de-5f6816d8f9de None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:59.524692 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 162/316] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 12:37:59.531255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f827d6-40c3-4992-9e7a-ce6ccb3a7420 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:59.532376 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f827d6-40c3-4992-9e7a-ce6ccb3a7420 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:59.532376 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f827d6-40c3-4992-9e7a-ce6ccb3a7420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:59.550508 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-66f827d6-40c3-4992-9e7a-ce6ccb3a7420 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:59.551658 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-66f827d6-40c3-4992-9e7a-ce6ccb3a7420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:59.596309 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9456ed9-e0ec-4004-9140-bb486811aa9e None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:59.635448 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55d1e6-ab8e-4689-b648-dd22010b6def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:59.636053 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 163/317] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 12:37:59.636711 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55d1e6-ab8e-4689-b648-dd22010b6def None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:59.637215 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55d1e6-ab8e-4689-b648-dd22010b6def None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:59.637765 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55d1e6-ab8e-4689-b648-dd22010b6def None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:59.652763 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f827d6-40c3-4992-9e7a-ce6ccb3a7420 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:59.654912 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:59.669403 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:59.731688 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b55d1e6-ab8e-4689-b648-dd22010b6def None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:59.733519 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b55d1e6-ab8e-4689-b648-dd22010b6def None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:59.764422 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f827d6-40c3-4992-9e7a-ce6ccb3a7420 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:59.805384 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b203f-7a4b-4959-940b-223bacc01f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:59.806758 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b203f-7a4b-4959-940b-223bacc01f96 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:59.807180 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b203f-7a4b-4959-940b-223bacc01f96 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:59.807661 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b203f-7a4b-4959-940b-223bacc01f96 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:59.808954 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4b203f-7a4b-4959-940b-223bacc01f96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:59.810704 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4b203f-7a4b-4959-940b-223bacc01f96 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:37:59.812752 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 164/318] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:37:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:37:59.814146 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d18cbb-c310-459b-be72-c5fcee407856 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:59.814421 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d18cbb-c310-459b-be72-c5fcee407856 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:59.814689 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d18cbb-c310-459b-be72-c5fcee407856 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:59.816097 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 165/319] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:37:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:37:59.821258 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d18cbb-c310-459b-be72-c5fcee407856 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:59.849747 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 155/320] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:37:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 12:37:59.851002 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107457fc-e596-4777-a136-cf431c64dc86 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:59.851299 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107457fc-e596-4777-a136-cf431c64dc86 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:59.851685 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107457fc-e596-4777-a136-cf431c64dc86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:59.870966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-107457fc-e596-4777-a136-cf431c64dc86 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:37:59.872196 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-107457fc-e596-4777-a136-cf431c64dc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:37:59.910246 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:59.931080 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:37:59.945268 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d18cbb-c310-459b-be72-c5fcee407856 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:37:59.955816 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 156/321] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:37:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:37:59.955816 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107457fc-e596-4777-a136-cf431c64dc86 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:37:59.988574 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da370d0a-286b-4fab-a964-cbfc13c2b78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:37:59.990534 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da370d0a-286b-4fab-a964-cbfc13c2b78c None admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:37:59.990534 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da370d0a-286b-4fab-a964-cbfc13c2b78c None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:37:59.990534 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da370d0a-286b-4fab-a964-cbfc13c2b78c None admin] PATH_INFO: `/v3/groups` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:37:59.990901 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da370d0a-286b-4fab-a964-cbfc13c2b78c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:37:59.992993 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da370d0a-286b-4fab-a964-cbfc13c2b78c None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:00.007927 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 166/322] 10.222.0.20 () {60 vars in 1134 bytes} [Tue Jul 26 12:37:59 2022] POST /identity/v3/groups => generated 226 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:38:00.013575 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da370d0a-286b-4fab-a964-cbfc13c2b78c None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:00.088181 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107457fc-e596-4777-a136-cf431c64dc86 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:00.134757 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5f2200-c0cd-42c1-97f3-5e93920bc330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:00.135422 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f2200-c0cd-42c1-97f3-5e93920bc330 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:00.135740 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f2200-c0cd-42c1-97f3-5e93920bc330 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:00.136170 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f2200-c0cd-42c1-97f3-5e93920bc330 None admin] PATH_INFO: `/v3/projects/eb19926b01264c0a80580d9e18a4397d` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:00.137844 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5f2200-c0cd-42c1-97f3-5e93920bc330 None admin] RBAC: Authorizing `identity:get_project(project_id=eb19926b01264c0a80580d9e18a4397d)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:00.139631 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5f2200-c0cd-42c1-97f3-5e93920bc330 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:00.142050 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 157/323] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:37:59 2022] GET /identity/v3/projects/eb19926b01264c0a80580d9e18a4397d => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:00.146040 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da370d0a-286b-4fab-a964-cbfc13c2b78c None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:00.148434 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5f2200-c0cd-42c1-97f3-5e93920bc330 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:00.180624 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:00.190487 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77537f36-c876-4568-89df-639ff5d5717d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:00.191538 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77537f36-c876-4568-89df-639ff5d5717d None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:00.192105 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77537f36-c876-4568-89df-639ff5d5717d None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:00.192663 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77537f36-c876-4568-89df-639ff5d5717d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:00.205457 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77537f36-c876-4568-89df-639ff5d5717d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:38:00.212038 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-77537f36-c876-4568-89df-639ff5d5717d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 12:38:00.212038 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:00.212038 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:00.212038 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:00.212038 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:00.212038 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:00.212038 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:00.212038 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:00.212038 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:00.212038 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:00.212038 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:00.212038 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:00.212038 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:38:00.212038 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:38:00.212038 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:38:00.212038 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:38:00.212941 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:38:00.212941 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:38:00.212941 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:38:00.212941 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:38:00.212941 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:00.212941 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:00.212941 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:00.212941 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:00.212941 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:00.212941 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:00.212941 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:00.212941 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:00.212941 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:00.212941 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:00.212941 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:00.212941 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:00.213735 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:00.213735 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:00.213735 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:00.213735 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:00.213735 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:38:00.213735 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:38:00.213735 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:38:00.213735 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:38:00.213735 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:38:00.213735 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:38:00.213735 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 12:38:00.213735 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:00.216155 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 167/324] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:38:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:00.222125 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77537f36-c876-4568-89df-639ff5d5717d None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:00.225050 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:00.291493 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5f2200-c0cd-42c1-97f3-5e93920bc330 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:00.336903 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7178c0ec-7c04-496a-a73b-8ff2c4df3f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:00.337623 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178c0ec-7c04-496a-a73b-8ff2c4df3f20 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:00.338016 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178c0ec-7c04-496a-a73b-8ff2c4df3f20 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:00.338404 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178c0ec-7c04-496a-a73b-8ff2c4df3f20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:00.339339 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7178c0ec-7c04-496a-a73b-8ff2c4df3f20 None admin] RBAC: Adding query filter params (role.id=db16c43424694218852e80f51fb1ee70, scope.project.id=eb19926b01264c0a80580d9e18a4397d, user.id=6703d260761041cb8af9777679b5b3ac) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:00.339755 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7178c0ec-7c04-496a-a73b-8ff2c4df3f20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:00.341130 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7178c0ec-7c04-496a-a73b-8ff2c4df3f20 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:00.350068 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 158/325] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:38:00 2022] GET /identity/v3/role_assignments?user.id=6703d260761041cb8af9777679b5b3ac&scope.project.id=eb19926b01264c0a80580d9e18a4397d&role.id=db16c43424694218852e80f51fb1ee70 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:00.362770 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77537f36-c876-4568-89df-639ff5d5717d None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:00.405413 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef114f5-13ee-41d9-a324-05916d81951f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:00.406448 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef114f5-13ee-41d9-a324-05916d81951f None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:00.406923 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef114f5-13ee-41d9-a324-05916d81951f None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:00.407201 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef114f5-13ee-41d9-a324-05916d81951f None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:00.408086 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef114f5-13ee-41d9-a324-05916d81951f None admin] RBAC: Adding query filter params (name=reader) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:00.408394 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef114f5-13ee-41d9-a324-05916d81951f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:00.409845 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef114f5-13ee-41d9-a324-05916d81951f None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:00.420285 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 168/326] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 26 12:38:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:00.425940 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7178c0ec-7c04-496a-a73b-8ff2c4df3f20 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:00.546471 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7178c0ec-7c04-496a-a73b-8ff2c4df3f20 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:00.565269 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:00.583289 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1922e83-5a45-4962-81fc-e1ff5a9dedd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:00.583976 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1922e83-5a45-4962-81fc-e1ff5a9dedd8 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:00.584262 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1922e83-5a45-4962-81fc-e1ff5a9dedd8 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:00.584531 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1922e83-5a45-4962-81fc-e1ff5a9dedd8 None admin] PATH_INFO: `/v3/users/5d01e35f85964c59a9ea5bb44bef9dbb` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:00.586393 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1922e83-5a45-4962-81fc-e1ff5a9dedd8 None admin] RBAC: Authorizing `identity:get_user(user_id=5d01e35f85964c59a9ea5bb44bef9dbb)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:00.587878 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1922e83-5a45-4962-81fc-e1ff5a9dedd8 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:00.589974 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 159/327] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:38:00 2022] GET /identity/v3/users/5d01e35f85964c59a9ea5bb44bef9dbb => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:00.595550 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef114f5-13ee-41d9-a324-05916d81951f None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:00.656905 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:00.713269 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef114f5-13ee-41d9-a324-05916d81951f None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:00.753057 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269adc69-99e9-4645-954d-1b9de46a3522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:00.753724 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269adc69-99e9-4645-954d-1b9de46a3522 None admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:00.754002 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269adc69-99e9-4645-954d-1b9de46a3522 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:00.754275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269adc69-99e9-4645-954d-1b9de46a3522 None admin] PATH_INFO: `/v3/system/users/5d01e35f85964c59a9ea5bb44bef9dbb/roles/db16c43424694218852e80f51fb1ee70` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:00.755794 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269adc69-99e9-4645-954d-1b9de46a3522 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5d01e35f85964c59a9ea5bb44bef9dbb, role_id=db16c43424694218852e80f51fb1ee70)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:00.757286 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269adc69-99e9-4645-954d-1b9de46a3522 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:00.770516 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 169/328] 10.222.0.20 () {58 vars in 1334 bytes} [Tue Jul 26 12:38:00 2022] PUT /identity/v3/system/users/5d01e35f85964c59a9ea5bb44bef9dbb/roles/db16c43424694218852e80f51fb1ee70 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:38:01.497832 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2cb69f-7e92-44a9-af3d-e78388d1b36f None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:01.498336 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2cb69f-7e92-44a9-af3d-e78388d1b36f None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:01.498411 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2cb69f-7e92-44a9-af3d-e78388d1b36f None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:01.499878 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 160/329] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:01.513081 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac72d8-4e66-48fd-bdce-21555f5d4924 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:01.513593 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac72d8-4e66-48fd-bdce-21555f5d4924 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:01.513593 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac72d8-4e66-48fd-bdce-21555f5d4924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:01.533560 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-52ac72d8-4e66-48fd-bdce-21555f5d4924 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:01.534771 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-52ac72d8-4e66-48fd-bdce-21555f5d4924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:01.628137 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 170/330] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:38:01.675091 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2cb69f-7e92-44a9-af3d-e78388d1b36f None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:01.795425 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2cb69f-7e92-44a9-af3d-e78388d1b36f None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:01.833959 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54762805-edee-4237-9b4d-ba026da9c93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:01.834614 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54762805-edee-4237-9b4d-ba026da9c93c None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:01.835464 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54762805-edee-4237-9b4d-ba026da9c93c None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:01.835761 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54762805-edee-4237-9b4d-ba026da9c93c None admin] PATH_INFO: `/v3/roles/member` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:01.856585 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54762805-edee-4237-9b4d-ba026da9c93c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:38:01.862105 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-54762805-edee-4237-9b4d-ba026da9c93c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 12:38:01.862105 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:01.862105 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:01.862105 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:01.862105 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:01.862105 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:01.862105 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:01.862105 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:01.862105 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:01.862105 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:01.862105 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:01.862105 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:01.862105 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:38:01.862105 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:38:01.862105 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:38:01.862105 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:38:01.862973 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:38:01.862973 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:38:01.862973 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:38:01.862973 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:38:01.862973 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:01.862973 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:01.862973 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:01.862973 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:01.862973 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:01.862973 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:01.862973 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:01.862973 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:01.862973 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:01.862973 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:01.862973 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:01.862973 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:01.863790 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:01.863790 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:01.863790 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:01.863790 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:01.863790 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:38:01.863790 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:38:01.863790 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:38:01.863790 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:38:01.863790 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:38:01.863790 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:38:01.863790 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 12:38:01.863790 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:01.863790 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 161/331] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:38:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:01.870971 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ac72d8-4e66-48fd-bdce-21555f5d4924 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:01.927102 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:02.000420 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ac72d8-4e66-48fd-bdce-21555f5d4924 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:02.048976 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf4b13-1959-48dd-9cd5-35747aaa4932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:02.049523 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf4b13-1959-48dd-9cd5-35747aaa4932 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:02.049803 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf4b13-1959-48dd-9cd5-35747aaa4932 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:02.050073 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf4b13-1959-48dd-9cd5-35747aaa4932 None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:02.050793 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf4b13-1959-48dd-9cd5-35747aaa4932 None admin] RBAC: Adding query filter params (name=member) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:02.051095 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf4b13-1959-48dd-9cd5-35747aaa4932 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:02.052503 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf4b13-1959-48dd-9cd5-35747aaa4932 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:02.067211 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 171/332] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 26 12:38:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:02.072474 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54762805-edee-4237-9b4d-ba026da9c93c None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:02.174749 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:02.258862 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54762805-edee-4237-9b4d-ba026da9c93c None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:02.311053 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad045b7-f065-4be7-8d31-d3ef2e461568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:02.311683 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad045b7-f065-4be7-8d31-d3ef2e461568 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:02.316302 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad045b7-f065-4be7-8d31-d3ef2e461568 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:02.316804 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad045b7-f065-4be7-8d31-d3ef2e461568 None admin] PATH_INFO: `/v3/users/b0b1eec2f7864118bb6144c335592d15` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:02.318270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad045b7-f065-4be7-8d31-d3ef2e461568 None admin] RBAC: Authorizing `identity:get_user(user_id=b0b1eec2f7864118bb6144c335592d15)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:02.320321 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad045b7-f065-4be7-8d31-d3ef2e461568 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:02.321969 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 162/333] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:38:02 2022] GET /identity/v3/users/b0b1eec2f7864118bb6144c335592d15 => generated 293 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:02.327937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf4b13-1959-48dd-9cd5-35747aaa4932 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:02.445146 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf4b13-1959-48dd-9cd5-35747aaa4932 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:02.454610 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:02.489959 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60735805-f038-48b9-953b-ca8bce34b973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:02.490586 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60735805-f038-48b9-953b-ca8bce34b973 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:02.490874 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60735805-f038-48b9-953b-ca8bce34b973 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:02.491198 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60735805-f038-48b9-953b-ca8bce34b973 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:02.492031 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60735805-f038-48b9-953b-ca8bce34b973 None admin] RBAC: Adding query filter params (role.id=01c9179cca2747a6b95b0decfca8b7ae, user.id=b0b1eec2f7864118bb6144c335592d15, scope.system=all) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:02.492338 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60735805-f038-48b9-953b-ca8bce34b973 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:02.493787 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60735805-f038-48b9-953b-ca8bce34b973 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:02.515012 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 172/334] 10.222.0.20 () {56 vars in 1308 bytes} [Tue Jul 26 12:38:02 2022] GET /identity/v3/role_assignments?user.id=b0b1eec2f7864118bb6144c335592d15&scope.system=all&role.id=01c9179cca2747a6b95b0decfca8b7ae => generated 510 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:03.242064 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2416bd8-4d39-4646-8a78-86af0fed1e94 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:03.242593 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2416bd8-4d39-4646-8a78-86af0fed1e94 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:03.242593 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2416bd8-4d39-4646-8a78-86af0fed1e94 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:03.244011 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 163/335] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:03.249713 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a46240-fb1a-4866-a3e8-ae42fbe21d31 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:03.250006 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a46240-fb1a-4866-a3e8-ae42fbe21d31 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:03.250288 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a46240-fb1a-4866-a3e8-ae42fbe21d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:03.276692 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-10a46240-fb1a-4866-a3e8-ae42fbe21d31 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:03.277905 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-10a46240-fb1a-4866-a3e8-ae42fbe21d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:03.287115 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91cedf5-b74a-41ff-9a4c-0d1a411b83df None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:03.287406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91cedf5-b74a-41ff-9a4c-0d1a411b83df None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:03.287692 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91cedf5-b74a-41ff-9a4c-0d1a411b83df None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:03.289407 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 164/336] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:03.300319 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d3a4e-42ce-4133-85ff-e8ea42a2743d None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:03.300608 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d3a4e-42ce-4133-85ff-e8ea42a2743d None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:03.300874 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d3a4e-42ce-4133-85ff-e8ea42a2743d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:03.331128 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-a39d3a4e-42ce-4133-85ff-e8ea42a2743d None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:03.332350 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-a39d3a4e-42ce-4133-85ff-e8ea42a2743d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:03.356643 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 173/337] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:38:03.381222 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a46240-fb1a-4866-a3e8-ae42fbe21d31 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:03.484351 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 165/338] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 12:38:03.496248 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:03.502053 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a46240-fb1a-4866-a3e8-ae42fbe21d31 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:03.520768 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39d3a4e-42ce-4133-85ff-e8ea42a2743d None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:03.564118 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae41193-f05e-4267-a9fb-4e192afac78b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:03.564789 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae41193-f05e-4267-a9fb-4e192afac78b None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:03.565045 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae41193-f05e-4267-a9fb-4e192afac78b None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:03.565317 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae41193-f05e-4267-a9fb-4e192afac78b None admin] PATH_INFO: `/v3/roles/member` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:03.596367 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae41193-f05e-4267-a9fb-4e192afac78b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:38:03.602440 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-9ae41193-f05e-4267-a9fb-4e192afac78b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 12:38:03.602440 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:03.602440 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:03.602440 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:03.602440 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:03.602440 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:03.602440 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:03.602440 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:03.602440 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:03.602440 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:03.602440 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:03.602440 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:03.602440 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:38:03.602440 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:38:03.602440 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:38:03.602440 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:38:03.603324 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:38:03.603324 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:38:03.603324 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:38:03.603324 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:38:03.603324 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:03.603324 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:03.603324 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:03.603324 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:03.603324 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:03.603324 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:03.603324 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:03.603324 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:03.603324 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:03.603324 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:03.603324 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:03.603324 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:03.604169 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:03.604169 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:03.604169 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:03.604169 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:03.604169 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:38:03.604169 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:38:03.604169 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:38:03.604169 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:38:03.604169 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:38:03.604169 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:38:03.604169 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 12:38:03.604169 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:03.613206 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 174/339] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:38:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:03.620282 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:03.621156 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae41193-f05e-4267-a9fb-4e192afac78b None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:03.709182 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39d3a4e-42ce-4133-85ff-e8ea42a2743d None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:03.754451 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd8f680-da67-41f6-92d7-41a3f14f5555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:03.758423 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd8f680-da67-41f6-92d7-41a3f14f5555 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:03.758423 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd8f680-da67-41f6-92d7-41a3f14f5555 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:03.758423 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd8f680-da67-41f6-92d7-41a3f14f5555 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:03.779747 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd8f680-da67-41f6-92d7-41a3f14f5555 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:38:03.784726 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-4bd8f680-da67-41f6-92d7-41a3f14f5555 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 12:38:03.784726 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:03.784726 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:03.784726 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:03.784726 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:03.784726 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:03.784726 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:03.784726 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:03.784726 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:03.784726 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:03.784726 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:03.784726 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:03.784726 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:38:03.784726 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:38:03.784726 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:38:03.784726 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:38:03.785599 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:38:03.785599 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:38:03.785599 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:38:03.785599 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:38:03.785599 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:03.785599 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:03.785599 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:03.785599 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:03.785599 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:03.785599 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:03.785599 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:03.785599 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:03.785599 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:03.785599 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:03.785599 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:03.785599 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:03.786874 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:03.786874 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:03.786874 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:03.786874 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:03.786874 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:38:03.786874 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:38:03.786874 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:38:03.786874 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:38:03.786874 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:38:03.786874 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:38:03.786874 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 12:38:03.786874 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:03.786874 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 166/340] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 26 12:38:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:38:03.797978 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd8f680-da67-41f6-92d7-41a3f14f5555 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:03.806130 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae41193-f05e-4267-a9fb-4e192afac78b None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:03.871810 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb084e0c-f0df-4a77-99e2-39d3277e34de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:03.871810 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb084e0c-f0df-4a77-99e2-39d3277e34de None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:03.871810 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb084e0c-f0df-4a77-99e2-39d3277e34de None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:03.871810 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb084e0c-f0df-4a77-99e2-39d3277e34de None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:03.872211 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb084e0c-f0df-4a77-99e2-39d3277e34de None admin] RBAC: Adding query filter params (name=member) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:03.872211 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb084e0c-f0df-4a77-99e2-39d3277e34de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:03.873585 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb084e0c-f0df-4a77-99e2-39d3277e34de None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:03.892279 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 175/341] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 26 12:38:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:38:03.899762 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb084e0c-f0df-4a77-99e2-39d3277e34de None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:03.942591 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:04.015315 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd8f680-da67-41f6-92d7-41a3f14f5555 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:04.022243 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:04.079711 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eb6db6-8a69-4e72-8f46-ba2ae1718ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:04.080665 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb6db6-8a69-4e72-8f46-ba2ae1718ab2 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:04.081380 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb6db6-8a69-4e72-8f46-ba2ae1718ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:04.081920 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb6db6-8a69-4e72-8f46-ba2ae1718ab2 None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:04.082791 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eb6db6-8a69-4e72-8f46-ba2ae1718ab2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:04.083239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eb6db6-8a69-4e72-8f46-ba2ae1718ab2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:04.086380 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eb6db6-8a69-4e72-8f46-ba2ae1718ab2 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:04.096796 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 167/342] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:38:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:38:04.098904 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088847a-d56c-42ab-93aa-9fba76fe36ff None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:04.099336 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088847a-d56c-42ab-93aa-9fba76fe36ff None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:04.099779 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088847a-d56c-42ab-93aa-9fba76fe36ff None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:04.101911 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 168/343] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:04.106650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6088847a-d56c-42ab-93aa-9fba76fe36ff None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:04.144796 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb084e0c-f0df-4a77-99e2-39d3277e34de None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:04.187468 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67167c68-84c6-4c3e-940a-f9b3c05d3c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:04.188325 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67167c68-84c6-4c3e-940a-f9b3c05d3c3b None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:04.188751 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67167c68-84c6-4c3e-940a-f9b3c05d3c3b None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:04.189207 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67167c68-84c6-4c3e-940a-f9b3c05d3c3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:04.249176 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6088847a-d56c-42ab-93aa-9fba76fe36ff None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:04.278398 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67167c68-84c6-4c3e-940a-f9b3c05d3c3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:04.280236 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67167c68-84c6-4c3e-940a-f9b3c05d3c3b None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:04.296367 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d77ac-40db-4e6b-b07a-190c8be94268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:04.297000 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d77ac-40db-4e6b-b07a-190c8be94268 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:04.297513 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d77ac-40db-4e6b-b07a-190c8be94268 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:04.297811 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d77ac-40db-4e6b-b07a-190c8be94268 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:04.302825 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:04.343672 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:04.391917 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50d77ac-40db-4e6b-b07a-190c8be94268 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:04.393652 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50d77ac-40db-4e6b-b07a-190c8be94268 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:04.401693 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 176/344] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:38:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 12:38:04.403332 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b586777-5cb0-4027-9bf5-bdd6bd136c3e None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:04.403853 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b586777-5cb0-4027-9bf5-bdd6bd136c3e None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:04.404330 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b586777-5cb0-4027-9bf5-bdd6bd136c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:04.423753 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-0b586777-5cb0-4027-9bf5-bdd6bd136c3e None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:04.425272 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-0b586777-5cb0-4027-9bf5-bdd6bd136c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:04.511082 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 177/345] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:38:04.512803 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3e2d8-c15f-4f49-b18b-8655f11d53e2 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:04.513297 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3e2d8-c15f-4f49-b18b-8655f11d53e2 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:04.513859 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3e2d8-c15f-4f49-b18b-8655f11d53e2 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:04.515676 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 178/346] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:04.516567 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a3e2d8-c15f-4f49-b18b-8655f11d53e2 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:04.523161 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 169/347] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:38:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 12:38:04.525848 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ec6d7-e8ab-4c24-8c05-67afce7e80d8 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:04.526149 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ec6d7-e8ab-4c24-8c05-67afce7e80d8 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:04.526423 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ec6d7-e8ab-4c24-8c05-67afce7e80d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:04.551017 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-4a3ec6d7-e8ab-4c24-8c05-67afce7e80d8 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:04.551895 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-4a3ec6d7-e8ab-4c24-8c05-67afce7e80d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:04.592073 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:04.658166 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:04.670979 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a3e2d8-c15f-4f49-b18b-8655f11d53e2 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:04.671727 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 170/348] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:38:04.672850 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ec6d7-e8ab-4c24-8c05-67afce7e80d8 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:04.736132 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df1b4ec-d10f-47f4-a1e1-7d0417f77f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:04.736787 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1b4ec-d10f-47f4-a1e1-7d0417f77f97 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:04.737071 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1b4ec-d10f-47f4-a1e1-7d0417f77f97 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:04.737344 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1b4ec-d10f-47f4-a1e1-7d0417f77f97 None admin] PATH_INFO: `/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:04.738603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df1b4ec-d10f-47f4-a1e1-7d0417f77f97 None admin] RBAC: Authorizing `identity:get_project(project_id=f5aedd2d5ee342aa80dfd96a062df0d9)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:04.740465 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df1b4ec-d10f-47f4-a1e1-7d0417f77f97 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:04.742595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 179/349] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:38:04 2022] GET /identity/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:04.743391 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df1b4ec-d10f-47f4-a1e1-7d0417f77f97 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:04.832125 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ec6d7-e8ab-4c24-8c05-67afce7e80d8 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:04.876758 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df1b4ec-d10f-47f4-a1e1-7d0417f77f97 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:04.877859 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e4711a-9d14-414b-a612-456f67ed709c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:04.880154 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4711a-9d14-414b-a612-456f67ed709c None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:04.880437 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4711a-9d14-414b-a612-456f67ed709c None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:04.880712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4711a-9d14-414b-a612-456f67ed709c None admin] PATH_INFO: `/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:04.882553 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e4711a-9d14-414b-a612-456f67ed709c None admin] RBAC: Authorizing `identity:get_project(project_id=f5aedd2d5ee342aa80dfd96a062df0d9)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:04.884070 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e4711a-9d14-414b-a612-456f67ed709c None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:04.886087 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 171/350] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:38:04 2022] GET /identity/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9 => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:04.887360 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e4711a-9d14-414b-a612-456f67ed709c None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:04.905065 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:04.929897 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d7d863-7312-41e6-bdf8-f666a73a01fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:04.930671 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d7d863-7312-41e6-bdf8-f666a73a01fc None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:04.931025 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d7d863-7312-41e6-bdf8-f666a73a01fc None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:04.931370 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d7d863-7312-41e6-bdf8-f666a73a01fc None admin] PATH_INFO: `/v3/roles/member` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:04.953470 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d7d863-7312-41e6-bdf8-f666a73a01fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:38:04.959704 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-34d7d863-7312-41e6-bdf8-f666a73a01fc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 12:38:04.959704 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:04.959704 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:04.959704 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:04.959704 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:04.959704 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:04.959704 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:04.959704 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:04.959704 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:04.959704 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:04.959704 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:04.959704 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:04.959704 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:38:04.959704 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:38:04.959704 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:38:04.959704 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:38:04.960662 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:38:04.960662 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:38:04.960662 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:38:04.960662 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:38:04.960662 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:04.960662 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:04.960662 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:04.960662 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:04.960662 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:04.960662 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:04.960662 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:04.960662 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:04.960662 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:04.960662 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:04.960662 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:04.960662 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:04.961473 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:04.961473 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:04.961473 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:04.961473 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:04.961473 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:38:04.961473 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:38:04.961473 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:38:04.961473 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:38:04.961473 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:38:04.961473 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:38:04.961473 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 12:38:04.961473 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:04.967738 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 180/351] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:38:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:04.969229 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239293e5-c5b5-4566-837e-77be5c84185f None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:04.969677 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239293e5-c5b5-4566-837e-77be5c84185f None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:04.970110 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239293e5-c5b5-4566-837e-77be5c84185f None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:04.971559 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 181/352] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:04.972065 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:04.972600 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239293e5-c5b5-4566-837e-77be5c84185f None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:05.024144 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e4711a-9d14-414b-a612-456f67ed709c None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:05.067299 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19ca0c9-346e-47b9-b0c8-0acfcfbd200a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:05.068244 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ca0c9-346e-47b9-b0c8-0acfcfbd200a None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:05.068700 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ca0c9-346e-47b9-b0c8-0acfcfbd200a None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:05.069155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ca0c9-346e-47b9-b0c8-0acfcfbd200a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:05.081417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19ca0c9-346e-47b9-b0c8-0acfcfbd200a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:38:05.087117 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-c19ca0c9-346e-47b9-b0c8-0acfcfbd200a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 12:38:05.087117 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:05.087117 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:05.087117 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:05.087117 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:05.087117 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:05.087117 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:05.087117 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:05.087117 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:05.087117 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:05.087117 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:05.087117 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:05.087117 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:38:05.087117 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:38:05.087117 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:38:05.087117 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:38:05.088046 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:38:05.088046 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:38:05.088046 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:38:05.088046 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:38:05.088046 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:05.088046 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:05.088046 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:05.088046 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:05.088046 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:05.088046 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:05.088046 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:05.088046 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:05.088046 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:05.088046 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:05.088046 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:05.088046 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:05.088827 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:05.088827 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:05.088827 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:05.088827 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:05.088827 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:38:05.088827 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:38:05.088827 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:38:05.088827 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:38:05.088827 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:38:05.088827 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:38:05.088827 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 12:38:05.088827 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:05.088827 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 172/353] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:38:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:05.089477 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19ca0c9-346e-47b9-b0c8-0acfcfbd200a None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:05.111834 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239293e5-c5b5-4566-837e-77be5c84185f None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:05.152517 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46bcd2f-0536-4912-8990-ba74c27788a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:05.153167 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46bcd2f-0536-4912-8990-ba74c27788a2 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:05.153446 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46bcd2f-0536-4912-8990-ba74c27788a2 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:05.153721 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46bcd2f-0536-4912-8990-ba74c27788a2 None admin] PATH_INFO: `/v3/groups/0869b90671e6441387d6cda07eeb9c4f` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:05.160326 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46bcd2f-0536-4912-8990-ba74c27788a2 None admin] RBAC: Authorizing `identity:get_group(group_id=0869b90671e6441387d6cda07eeb9c4f)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:05.161806 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46bcd2f-0536-4912-8990-ba74c27788a2 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:05.163564 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 182/354] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Jul 26 12:38:04 2022] GET /identity/v3/groups/0869b90671e6441387d6cda07eeb9c4f => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:05.169195 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46bcd2f-0536-4912-8990-ba74c27788a2 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:05.223320 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19ca0c9-346e-47b9-b0c8-0acfcfbd200a None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:05.228521 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:05.251208 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:05.267821 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43664d-97db-49fc-8a7b-15a5e44fd3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:05.268488 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43664d-97db-49fc-8a7b-15a5e44fd3e0 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:05.268773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43664d-97db-49fc-8a7b-15a5e44fd3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:05.269051 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43664d-97db-49fc-8a7b-15a5e44fd3e0 None admin] PATH_INFO: `/v3/groups/0869b90671e6441387d6cda07eeb9c4f` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:05.270694 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f43664d-97db-49fc-8a7b-15a5e44fd3e0 None admin] RBAC: Authorizing `identity:get_group(group_id=0869b90671e6441387d6cda07eeb9c4f)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:05.272022 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f43664d-97db-49fc-8a7b-15a5e44fd3e0 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:05.273898 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 173/355] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Jul 26 12:38:05 2022] GET /identity/v3/groups/0869b90671e6441387d6cda07eeb9c4f => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:05.274950 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc935c11-4d17-4d1e-9afe-ddbdce4951c9 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:05.275231 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc935c11-4d17-4d1e-9afe-ddbdce4951c9 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:05.275511 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc935c11-4d17-4d1e-9afe-ddbdce4951c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:05.296398 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-fc935c11-4d17-4d1e-9afe-ddbdce4951c9 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:05.297630 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-fc935c11-4d17-4d1e-9afe-ddbdce4951c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:05.299767 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46bcd2f-0536-4912-8990-ba74c27788a2 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:05.338415 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dadab2-a6ec-4bc4-af95-6e174653978c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:05.338969 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dadab2-a6ec-4bc4-af95-6e174653978c None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:05.339251 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dadab2-a6ec-4bc4-af95-6e174653978c None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:05.339527 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dadab2-a6ec-4bc4-af95-6e174653978c None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:05.340405 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dadab2-a6ec-4bc4-af95-6e174653978c None admin] RBAC: Adding query filter params (name=member) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:05.341122 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dadab2-a6ec-4bc4-af95-6e174653978c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:05.342111 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dadab2-a6ec-4bc4-af95-6e174653978c None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:05.351473 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 183/356] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 26 12:38:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:05.352328 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dadab2-a6ec-4bc4-af95-6e174653978c None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:05.374929 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 174/357] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:38:05.376252 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a7928-0d8c-4938-b7e7-323e3bae8b7d None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:05.376544 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a7928-0d8c-4938-b7e7-323e3bae8b7d None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:05.376818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a7928-0d8c-4938-b7e7-323e3bae8b7d None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:05.378215 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 175/358] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:05.378776 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a7928-0d8c-4938-b7e7-323e3bae8b7d None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:05.473309 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dadab2-a6ec-4bc4-af95-6e174653978c None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:05.475200 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:05.510503 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a7928-0d8c-4938-b7e7-323e3bae8b7d None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:05.522020 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1966d4-c7db-403f-b695-452a52867375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:05.523695 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1966d4-c7db-403f-b695-452a52867375 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:05.523983 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1966d4-c7db-403f-b695-452a52867375 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:05.524264 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1966d4-c7db-403f-b695-452a52867375 None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:05.525396 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1966d4-c7db-403f-b695-452a52867375 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:05.525705 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1966d4-c7db-403f-b695-452a52867375 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:05.527173 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1966d4-c7db-403f-b695-452a52867375 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:05.542783 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 184/359] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 26 12:38:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:05.545165 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1966d4-c7db-403f-b695-452a52867375 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:05.546825 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:05.565965 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a132c9a3-bd8b-45b5-aff1-96d646c8e69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:05.566905 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132c9a3-bd8b-45b5-aff1-96d646c8e69f None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:05.567192 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132c9a3-bd8b-45b5-aff1-96d646c8e69f None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:05.567467 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132c9a3-bd8b-45b5-aff1-96d646c8e69f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:05.568461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a132c9a3-bd8b-45b5-aff1-96d646c8e69f None admin] RBAC: Adding query filter params (group.id=0869b90671e6441387d6cda07eeb9c4f, role.id=01c9179cca2747a6b95b0decfca8b7ae, scope.project.id=f5aedd2d5ee342aa80dfd96a062df0d9) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:05.569304 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a132c9a3-bd8b-45b5-aff1-96d646c8e69f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:05.570704 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a132c9a3-bd8b-45b5-aff1-96d646c8e69f None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:05.577515 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 176/360] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Jul 26 12:38:05 2022] GET /identity/v3/role_assignments?group.id=0869b90671e6441387d6cda07eeb9c4f&scope.project.id=f5aedd2d5ee342aa80dfd96a062df0d9&role.id=01c9179cca2747a6b95b0decfca8b7ae => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:05.578619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a132c9a3-bd8b-45b5-aff1-96d646c8e69f None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:05.705506 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1966d4-c7db-403f-b695-452a52867375 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:05.725783 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a132c9a3-bd8b-45b5-aff1-96d646c8e69f None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:05.758984 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb67bc0-b926-47c6-a1a2-ac55c56b16df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:05.759697 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb67bc0-b926-47c6-a1a2-ac55c56b16df None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:05.762210 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb67bc0-b926-47c6-a1a2-ac55c56b16df None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:05.762210 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb67bc0-b926-47c6-a1a2-ac55c56b16df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:05.762649 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb67bc0-b926-47c6-a1a2-ac55c56b16df None admin] RBAC: Adding query filter params (group.id=0869b90671e6441387d6cda07eeb9c4f, role.id=6589779ddd834447a1ed1921e1c834f6, scope.project.id=f5aedd2d5ee342aa80dfd96a062df0d9) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:05.762801 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb67bc0-b926-47c6-a1a2-ac55c56b16df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:05.764287 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb67bc0-b926-47c6-a1a2-ac55c56b16df None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:05.771930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 185/361] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Jul 26 12:38:05 2022] GET /identity/v3/role_assignments?group.id=0869b90671e6441387d6cda07eeb9c4f&scope.project.id=f5aedd2d5ee342aa80dfd96a062df0d9&role.id=6589779ddd834447a1ed1921e1c834f6 => generated 260 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:05.773231 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7c7658-626c-44ca-ad05-19a8bb10c2fe None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:05.773346 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275bfc37-f8b1-4d4b-b957-b5f543830567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:05.773519 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7c7658-626c-44ca-ad05-19a8bb10c2fe None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:05.773788 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7c7658-626c-44ca-ad05-19a8bb10c2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:05.773892 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275bfc37-f8b1-4d4b-b957-b5f543830567 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:05.774174 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275bfc37-f8b1-4d4b-b957-b5f543830567 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:05.774445 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275bfc37-f8b1-4d4b-b957-b5f543830567 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:05.794740 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-4e7c7658-626c-44ca-ad05-19a8bb10c2fe None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:05.795986 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-4e7c7658-626c-44ca-ad05-19a8bb10c2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:05.833951 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:05.847012 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:05.863463 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275bfc37-f8b1-4d4b-b957-b5f543830567 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:05.865250 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275bfc37-f8b1-4d4b-b957-b5f543830567 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:05.890538 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 186/362] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:38:05.892000 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7c7658-626c-44ca-ad05-19a8bb10c2fe None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:06.000695 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 177/363] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:38:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 12:38:06.001589 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275bfc37-f8b1-4d4b-b957-b5f543830567 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:06.018595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7c7658-626c-44ca-ad05-19a8bb10c2fe None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:06.067385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ac93b7-6626-481a-b054-addf9b6178aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:06.068353 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac93b7-6626-481a-b054-addf9b6178aa None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:06.068699 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac93b7-6626-481a-b054-addf9b6178aa None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:06.068977 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac93b7-6626-481a-b054-addf9b6178aa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:06.082330 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ac93b7-6626-481a-b054-addf9b6178aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:38:06.087461 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-44ac93b7-6626-481a-b054-addf9b6178aa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 12:38:06.087461 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:06.087461 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:06.087461 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:06.087461 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:06.087461 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:06.087461 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:06.087461 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:06.087461 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:06.087461 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:06.087461 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:06.087461 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:06.087461 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:38:06.087461 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:38:06.087461 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:38:06.087461 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:38:06.088438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:38:06.088438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:38:06.088438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:38:06.088438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:38:06.088438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:06.088438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:06.088438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:06.088438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:06.088438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:06.088438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:06.088438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:06.088438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:06.088438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:06.088438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:06.088438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:06.088438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:06.089276 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:06.089276 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:06.089276 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:06.089276 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:06.089276 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:38:06.089276 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:38:06.089276 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:38:06.089276 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:38:06.089276 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:38:06.089276 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:38:06.089276 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 12:38:06.089276 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:06.089276 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 187/364] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 26 12:38:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:06.089975 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ac93b7-6626-481a-b054-addf9b6178aa None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:06.116041 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:06.117618 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:06.135222 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275bfc37-f8b1-4d4b-b957-b5f543830567 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:06.181052 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee690b9e-91e7-4d13-bbca-f5666db0dfd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:06.182408 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee690b9e-91e7-4d13-bbca-f5666db0dfd4 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:06.182703 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee690b9e-91e7-4d13-bbca-f5666db0dfd4 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:06.182977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee690b9e-91e7-4d13-bbca-f5666db0dfd4 None admin] PATH_INFO: `/v3/users/5d01e35f85964c59a9ea5bb44bef9dbb` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:06.184214 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee690b9e-91e7-4d13-bbca-f5666db0dfd4 None admin] RBAC: Authorizing `identity:get_user(user_id=5d01e35f85964c59a9ea5bb44bef9dbb)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:06.186596 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee690b9e-91e7-4d13-bbca-f5666db0dfd4 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:06.188472 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 178/365] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:38:06 2022] GET /identity/v3/users/5d01e35f85964c59a9ea5bb44bef9dbb => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:06.189242 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee690b9e-91e7-4d13-bbca-f5666db0dfd4 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:06.237895 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ac93b7-6626-481a-b054-addf9b6178aa None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:06.274801 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bcb630-1af9-420d-a1d4-6a665ff50322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:06.275364 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bcb630-1af9-420d-a1d4-6a665ff50322 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:06.275660 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bcb630-1af9-420d-a1d4-6a665ff50322 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:06.275965 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bcb630-1af9-420d-a1d4-6a665ff50322 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:06.286894 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bcb630-1af9-420d-a1d4-6a665ff50322 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:38:06.293597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-d8bcb630-1af9-420d-a1d4-6a665ff50322 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:38:06.293597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:06.293597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:06.293597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:06.293597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:06.293597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:06.293597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:06.293597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:06.293597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:06.293597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:06.293597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:06.293597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:06.293597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:38:06.293597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:38:06.293597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:38:06.293597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:38:06.294492 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:38:06.294492 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:38:06.294492 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:38:06.294492 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:38:06.294492 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:06.294492 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:06.294492 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:06.294492 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:06.294492 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:06.294492 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:06.294492 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:06.294492 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:06.294492 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:06.294492 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:06.294492 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:06.294492 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:06.295280 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:06.295280 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:06.295280 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:06.295280 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:06.295280 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:38:06.295280 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:38:06.295280 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:38:06.295280 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:38:06.295280 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:38:06.295280 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:38:06.295280 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:38:06.295280 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:06.295280 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 188/366] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 26 12:38:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:06.295996 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bcb630-1af9-420d-a1d4-6a665ff50322 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:06.318004 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee690b9e-91e7-4d13-bbca-f5666db0dfd4 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:06.355862 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6313f3ac-8a2f-496b-9573-87fd4652ba3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:06.357979 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313f3ac-8a2f-496b-9573-87fd4652ba3b None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:06.358267 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313f3ac-8a2f-496b-9573-87fd4652ba3b None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:06.358650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313f3ac-8a2f-496b-9573-87fd4652ba3b None admin] PATH_INFO: `/v3/projects/eb19926b01264c0a80580d9e18a4397d` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:06.359957 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6313f3ac-8a2f-496b-9573-87fd4652ba3b None admin] RBAC: Authorizing `identity:get_project(project_id=eb19926b01264c0a80580d9e18a4397d)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:06.361508 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6313f3ac-8a2f-496b-9573-87fd4652ba3b None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:06.363815 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 179/367] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:38:06 2022] GET /identity/v3/projects/eb19926b01264c0a80580d9e18a4397d => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:06.364943 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6313f3ac-8a2f-496b-9573-87fd4652ba3b None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:06.384926 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:06.398991 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:06.412020 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bcb630-1af9-420d-a1d4-6a665ff50322 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:06.454849 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8ad264-632b-4b5e-b806-88aea3908676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:06.455527 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ad264-632b-4b5e-b806-88aea3908676 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:06.455838 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ad264-632b-4b5e-b806-88aea3908676 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:06.456108 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ad264-632b-4b5e-b806-88aea3908676 None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:06.456921 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8ad264-632b-4b5e-b806-88aea3908676 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:06.457227 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8ad264-632b-4b5e-b806-88aea3908676 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:06.458619 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8ad264-632b-4b5e-b806-88aea3908676 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:06.469165 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 189/368] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:38:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:06.469783 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8ad264-632b-4b5e-b806-88aea3908676 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:06.515373 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6313f3ac-8a2f-496b-9573-87fd4652ba3b None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:06.566027 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a2dd1-5326-4ed2-85ec-fa4ea16f0482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:06.567125 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a2dd1-5326-4ed2-85ec-fa4ea16f0482 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:06.567627 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a2dd1-5326-4ed2-85ec-fa4ea16f0482 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:06.568453 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a2dd1-5326-4ed2-85ec-fa4ea16f0482 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:06.570942 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270a2dd1-5326-4ed2-85ec-fa4ea16f0482 None admin] RBAC: Adding query filter params (role.id=db16c43424694218852e80f51fb1ee70, user.id=5d01e35f85964c59a9ea5bb44bef9dbb, scope.system=all) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:06.571479 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270a2dd1-5326-4ed2-85ec-fa4ea16f0482 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:06.573194 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270a2dd1-5326-4ed2-85ec-fa4ea16f0482 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:06.590262 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 180/369] 10.222.0.20 () {56 vars in 1308 bytes} [Tue Jul 26 12:38:06 2022] GET /identity/v3/role_assignments?user.id=5d01e35f85964c59a9ea5bb44bef9dbb&scope.system=all&role.id=db16c43424694218852e80f51fb1ee70 => generated 510 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:06.592024 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a2dd1-5326-4ed2-85ec-fa4ea16f0482 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:06.614897 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8ad264-632b-4b5e-b806-88aea3908676 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:06.665342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e9900c-1c73-4d69-9176-0ba4990b26f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:06.666685 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9900c-1c73-4d69-9176-0ba4990b26f5 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:06.667164 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9900c-1c73-4d69-9176-0ba4990b26f5 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:06.667621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9900c-1c73-4d69-9176-0ba4990b26f5 None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:06.668586 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e9900c-1c73-4d69-9176-0ba4990b26f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:06.669076 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e9900c-1c73-4d69-9176-0ba4990b26f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:06.671365 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e9900c-1c73-4d69-9176-0ba4990b26f5 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:06.684783 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 190/370] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:38:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:06.685604 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e9900c-1c73-4d69-9176-0ba4990b26f5 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:06.689370 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:06.713649 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:06.745819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a2dd1-5326-4ed2-85ec-fa4ea16f0482 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:06.837328 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e9900c-1c73-4d69-9176-0ba4990b26f5 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:06.839102 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9744cc42-2f73-402e-92b6-7561ee509997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:06.839786 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744cc42-2f73-402e-92b6-7561ee509997 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:06.840093 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744cc42-2f73-402e-92b6-7561ee509997 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:06.840456 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744cc42-2f73-402e-92b6-7561ee509997 None admin] PATH_INFO: `/v3/groups/0869b90671e6441387d6cda07eeb9c4f` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:06.842313 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9744cc42-2f73-402e-92b6-7561ee509997 None admin] RBAC: Authorizing `identity:get_group(group_id=0869b90671e6441387d6cda07eeb9c4f)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:06.843074 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9744cc42-2f73-402e-92b6-7561ee509997 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:06.845029 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 181/371] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Jul 26 12:38:06 2022] GET /identity/v3/groups/0869b90671e6441387d6cda07eeb9c4f => generated 226 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:06.845814 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9744cc42-2f73-402e-92b6-7561ee509997 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:06.896969 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da7c91-0318-490c-94cc-e659d96f8e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:06.898741 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da7c91-0318-490c-94cc-e659d96f8e06 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:06.899373 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da7c91-0318-490c-94cc-e659d96f8e06 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:06.899948 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da7c91-0318-490c-94cc-e659d96f8e06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:06.984437 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38da7c91-0318-490c-94cc-e659d96f8e06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:06.986095 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38da7c91-0318-490c-94cc-e659d96f8e06 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:06.990541 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:07.000602 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9744cc42-2f73-402e-92b6-7561ee509997 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:07.046730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c520b7d-5da6-4fb7-9b73-2a42e4004a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:07.047369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c520b7d-5da6-4fb7-9b73-2a42e4004a69 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:07.047667 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c520b7d-5da6-4fb7-9b73-2a42e4004a69 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:07.047971 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c520b7d-5da6-4fb7-9b73-2a42e4004a69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:07.074386 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:07.128862 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 191/372] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:38:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 12:38:07.130173 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da7c91-0318-490c-94cc-e659d96f8e06 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:07.144141 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c520b7d-5da6-4fb7-9b73-2a42e4004a69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:07.145611 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c520b7d-5da6-4fb7-9b73-2a42e4004a69 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:07.255904 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da7c91-0318-490c-94cc-e659d96f8e06 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:07.257651 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:07.275360 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 182/373] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:38:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 12:38:07.276276 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c520b7d-5da6-4fb7-9b73-2a42e4004a69 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:07.297259 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241512f3-6c77-427d-9f70-5574d2791e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:07.297883 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241512f3-6c77-427d-9f70-5574d2791e01 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:07.298161 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241512f3-6c77-427d-9f70-5574d2791e01 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:07.298434 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241512f3-6c77-427d-9f70-5574d2791e01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:07.299330 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241512f3-6c77-427d-9f70-5574d2791e01 None admin] RBAC: Adding query filter params (group.id=0869b90671e6441387d6cda07eeb9c4f, role.id=01c9179cca2747a6b95b0decfca8b7ae, scope.project.id=eb19926b01264c0a80580d9e18a4397d) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:07.299662 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241512f3-6c77-427d-9f70-5574d2791e01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:07.301115 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241512f3-6c77-427d-9f70-5574d2791e01 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:07.308141 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 192/374] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Jul 26 12:38:07 2022] GET /identity/v3/role_assignments?group.id=0869b90671e6441387d6cda07eeb9c4f&scope.project.id=eb19926b01264c0a80580d9e18a4397d&role.id=01c9179cca2747a6b95b0decfca8b7ae => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:07.309044 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241512f3-6c77-427d-9f70-5574d2791e01 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:07.361810 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:07.426108 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241512f3-6c77-427d-9f70-5574d2791e01 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:07.464904 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fec5b1-0b45-4e86-8b7b-d5278758ccb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:07.465548 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fec5b1-0b45-4e86-8b7b-d5278758ccb8 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:07.465827 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fec5b1-0b45-4e86-8b7b-d5278758ccb8 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:07.466102 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fec5b1-0b45-4e86-8b7b-d5278758ccb8 None admin] PATH_INFO: `/v3/projects/1ca671277c17458082e4383a242d5404` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:07.467312 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fec5b1-0b45-4e86-8b7b-d5278758ccb8 None admin] RBAC: Authorizing `identity:get_project(project_id=1ca671277c17458082e4383a242d5404)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:07.468841 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fec5b1-0b45-4e86-8b7b-d5278758ccb8 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:07.470682 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 193/375] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:38:07 2022] GET /identity/v3/projects/1ca671277c17458082e4383a242d5404 => generated 344 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:07.477104 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fec5b1-0b45-4e86-8b7b-d5278758ccb8 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:07.517158 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:07.540552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c520b7d-5da6-4fb7-9b73-2a42e4004a69 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:07.596507 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b15272c-3567-4d3b-8fa4-f8ae776fedc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:07.597326 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b15272c-3567-4d3b-8fa4-f8ae776fedc9 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:07.597936 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b15272c-3567-4d3b-8fa4-f8ae776fedc9 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:07.598220 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b15272c-3567-4d3b-8fa4-f8ae776fedc9 None admin] PATH_INFO: `/v3/projects/eb19926b01264c0a80580d9e18a4397d` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:07.599722 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b15272c-3567-4d3b-8fa4-f8ae776fedc9 None admin] RBAC: Authorizing `identity:get_project(project_id=eb19926b01264c0a80580d9e18a4397d)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:07.601119 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fec5b1-0b45-4e86-8b7b-d5278758ccb8 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:07.601716 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b15272c-3567-4d3b-8fa4-f8ae776fedc9 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:07.613241 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 183/376] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:38:07 2022] GET /identity/v3/projects/eb19926b01264c0a80580d9e18a4397d => generated 302 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:07.619129 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b15272c-3567-4d3b-8fa4-f8ae776fedc9 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:07.681935 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26a001-7b89-402a-9900-6ab093d56784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:07.682519 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26a001-7b89-402a-9900-6ab093d56784 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:07.682798 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26a001-7b89-402a-9900-6ab093d56784 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:07.683071 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26a001-7b89-402a-9900-6ab093d56784 None admin] PATH_INFO: `/v3/groups/c31a89a75973498f87919d406a571a79` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:07.691628 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26a001-7b89-402a-9900-6ab093d56784 None admin] RBAC: Authorizing `identity:get_group(group_id=c31a89a75973498f87919d406a571a79)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:07.693149 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26a001-7b89-402a-9900-6ab093d56784 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:07.695415 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 194/377] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Jul 26 12:38:07 2022] GET /identity/v3/groups/c31a89a75973498f87919d406a571a79 => generated 229 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:38:07.703001 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26a001-7b89-402a-9900-6ab093d56784 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:07.812468 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b15272c-3567-4d3b-8fa4-f8ae776fedc9 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:07.846214 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:07.874670 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26a001-7b89-402a-9900-6ab093d56784 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:07.877320 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:07.887914 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7cadf-628b-4e29-a3d7-d69db089cf53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:07.888754 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7cadf-628b-4e29-a3d7-d69db089cf53 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:07.889233 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7cadf-628b-4e29-a3d7-d69db089cf53 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:07.889611 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7cadf-628b-4e29-a3d7-d69db089cf53 None admin] PATH_INFO: `/v3/groups/0869b90671e6441387d6cda07eeb9c4f` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:07.895044 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad7cadf-628b-4e29-a3d7-d69db089cf53 None admin] RBAC: Authorizing `identity:get_group(group_id=0869b90671e6441387d6cda07eeb9c4f)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:07.896596 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad7cadf-628b-4e29-a3d7-d69db089cf53 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:07.898946 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 184/378] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Jul 26 12:38:07 2022] GET /identity/v3/groups/0869b90671e6441387d6cda07eeb9c4f => generated 226 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:38:07.904879 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7cadf-628b-4e29-a3d7-d69db089cf53 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:07.920831 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c9bc2-1eff-4271-9d61-3c4a717a6cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:07.921462 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c9bc2-1eff-4271-9d61-3c4a717a6cd8 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:07.921746 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c9bc2-1eff-4271-9d61-3c4a717a6cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:07.922020 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c9bc2-1eff-4271-9d61-3c4a717a6cd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:07.922821 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6c9bc2-1eff-4271-9d61-3c4a717a6cd8 None admin] RBAC: Adding query filter params (group.id=c31a89a75973498f87919d406a571a79, role.id=0e78444e43c744668178971af4087ca0, scope.project.id=1ca671277c17458082e4383a242d5404) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:07.923132 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6c9bc2-1eff-4271-9d61-3c4a717a6cd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:07.924618 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6c9bc2-1eff-4271-9d61-3c4a717a6cd8 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:07.939914 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 195/379] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Jul 26 12:38:07 2022] GET /identity/v3/role_assignments?group.id=c31a89a75973498f87919d406a571a79&scope.project.id=1ca671277c17458082e4383a242d5404&role.id=0e78444e43c744668178971af4087ca0 => generated 260 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:07.944315 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8e28c-a7e0-481f-9fb1-0d652b2b042c None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:07.944315 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8e28c-a7e0-481f-9fb1-0d652b2b042c None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:07.944315 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8e28c-a7e0-481f-9fb1-0d652b2b042c None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:07.944643 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 196/380] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 12:38:07.950348 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a741e-a602-4773-8e8c-968338c4dcb8 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:07.950662 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a741e-a602-4773-8e8c-968338c4dcb8 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:07.950908 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a741e-a602-4773-8e8c-968338c4dcb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:07.984059 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-bb8a741e-a602-4773-8e8c-968338c4dcb8 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:07.985323 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-bb8a741e-a602-4773-8e8c-968338c4dcb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:08.069260 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7cadf-628b-4e29-a3d7-d69db089cf53 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:08.117085 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c0ffd-5790-4b80-a274-563cb3523e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:08.118208 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c0ffd-5790-4b80-a274-563cb3523e2c None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:08.118504 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c0ffd-5790-4b80-a274-563cb3523e2c None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:08.118787 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c0ffd-5790-4b80-a274-563cb3523e2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:08.119904 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18c0ffd-5790-4b80-a274-563cb3523e2c None admin] RBAC: Adding query filter params (group.id=0869b90671e6441387d6cda07eeb9c4f, role.id=6589779ddd834447a1ed1921e1c834f6, scope.project.id=eb19926b01264c0a80580d9e18a4397d) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:08.120264 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18c0ffd-5790-4b80-a274-563cb3523e2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:08.123942 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18c0ffd-5790-4b80-a274-563cb3523e2c None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:08.129422 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 185/381] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Jul 26 12:38:07 2022] GET /identity/v3/role_assignments?group.id=0869b90671e6441387d6cda07eeb9c4f&scope.project.id=eb19926b01264c0a80580d9e18a4397d&role.id=6589779ddd834447a1ed1921e1c834f6 => generated 260 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:38:08.136198 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 197/382] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 12:38:08.164570 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c0ffd-5790-4b80-a274-563cb3523e2c None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:08.222601 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:08.281091 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c0ffd-5790-4b80-a274-563cb3523e2c None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:08.318392 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b638ff-3e8c-445b-afbf-194764bb2aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:08.319564 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b638ff-3e8c-445b-afbf-194764bb2aeb None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:08.319959 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b638ff-3e8c-445b-afbf-194764bb2aeb None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:08.320235 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b638ff-3e8c-445b-afbf-194764bb2aeb None admin] PATH_INFO: `/v3/roles/member` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:08.331947 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b638ff-3e8c-445b-afbf-194764bb2aeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:38:08.337751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-97b638ff-3e8c-445b-afbf-194764bb2aeb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 12:38:08.337751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:08.337751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:08.337751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:08.337751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:08.337751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:08.337751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:08.337751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:08.337751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:08.337751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:08.337751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:08.337751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:08.337751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:38:08.337751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:38:08.337751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:38:08.337751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:38:08.338702 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:38:08.338702 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:38:08.338702 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:38:08.338702 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:38:08.338702 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:08.338702 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:08.338702 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:08.338702 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:08.338702 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:08.338702 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:08.338702 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:08.338702 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:08.338702 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:08.338702 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:08.338702 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:08.338702 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:08.339491 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:08.339491 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:08.339491 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:08.339491 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:08.339491 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:38:08.339491 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:38:08.339491 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:38:08.339491 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:38:08.339491 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:38:08.339491 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:38:08.339491 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 12:38:08.339491 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:08.343785 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 186/383] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:38:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:08.345934 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8a741e-a602-4773-8e8c-968338c4dcb8 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:08.402745 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0f7ce-0e93-4e6f-a671-9bae51d6d528 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:08.402745 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0f7ce-0e93-4e6f-a671-9bae51d6d528 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:08.403343 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0f7ce-0e93-4e6f-a671-9bae51d6d528 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:08.406023 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 187/384] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:08.411135 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf43e6-e666-45dc-922a-a3aefc01afa9 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:08.411522 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf43e6-e666-45dc-922a-a3aefc01afa9 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:08.411899 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf43e6-e666-45dc-922a-a3aefc01afa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:08.438277 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-cadf43e6-e666-45dc-922a-a3aefc01afa9 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:08.440225 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-cadf43e6-e666-45dc-922a-a3aefc01afa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:08.499481 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:08.507822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8a741e-a602-4773-8e8c-968338c4dcb8 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:08.528086 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 188/385] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 12:38:08.551772 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf43e6-e666-45dc-922a-a3aefc01afa9 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:08.571958 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a80b72-39c5-4b9b-9a19-ff84e57db3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:08.572865 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a80b72-39c5-4b9b-9a19-ff84e57db3f7 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:08.573306 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a80b72-39c5-4b9b-9a19-ff84e57db3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:08.573734 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a80b72-39c5-4b9b-9a19-ff84e57db3f7 None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:08.574604 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a80b72-39c5-4b9b-9a19-ff84e57db3f7 None admin] RBAC: Adding query filter params (name=member) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:08.575056 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a80b72-39c5-4b9b-9a19-ff84e57db3f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:08.577565 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a80b72-39c5-4b9b-9a19-ff84e57db3f7 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:08.594918 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 198/386] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 26 12:38:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:38:08.601427 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a80b72-39c5-4b9b-9a19-ff84e57db3f7 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:08.630460 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:08.761777 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a80b72-39c5-4b9b-9a19-ff84e57db3f7 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:08.797383 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf43e6-e666-45dc-922a-a3aefc01afa9 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:08.804352 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53beb8a7-e0bf-466d-8b8f-2c0712e7d33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:08.805008 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53beb8a7-e0bf-466d-8b8f-2c0712e7d33a None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:08.805287 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53beb8a7-e0bf-466d-8b8f-2c0712e7d33a None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:08.805890 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53beb8a7-e0bf-466d-8b8f-2c0712e7d33a None admin] PATH_INFO: `/v3/groups/0869b90671e6441387d6cda07eeb9c4f` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:08.807448 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53beb8a7-e0bf-466d-8b8f-2c0712e7d33a None admin] RBAC: Authorizing `identity:get_group(group_id=0869b90671e6441387d6cda07eeb9c4f)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:08.808952 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53beb8a7-e0bf-466d-8b8f-2c0712e7d33a None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:08.810622 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 199/387] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Jul 26 12:38:08 2022] GET /identity/v3/groups/0869b90671e6441387d6cda07eeb9c4f => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:08.821635 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53beb8a7-e0bf-466d-8b8f-2c0712e7d33a None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:08.850977 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:08.860381 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad2e692-b4bd-4c69-9e81-c4e147124704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:08.861119 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad2e692-b4bd-4c69-9e81-c4e147124704 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:08.861406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad2e692-b4bd-4c69-9e81-c4e147124704 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:08.861683 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad2e692-b4bd-4c69-9e81-c4e147124704 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:08.881020 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad2e692-b4bd-4c69-9e81-c4e147124704 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:38:08.892830 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-4ad2e692-b4bd-4c69-9e81-c4e147124704 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 12:38:08.892830 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:08.892830 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:08.892830 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:08.892830 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:08.892830 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:08.892830 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:08.892830 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:08.892830 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:08.892830 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:08.892830 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:08.892830 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:08.892830 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:38:08.892830 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:38:08.892830 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:38:08.892830 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:38:08.893799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:38:08.893799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:38:08.893799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:38:08.893799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:38:08.893799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:08.893799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:08.893799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:08.893799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:08.893799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:08.893799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:08.893799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:08.893799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:08.893799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:08.893799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:08.893799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:08.893799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:08.894600 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:08.894600 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:08.894600 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:08.894600 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:08.894600 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:38:08.894600 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:38:08.894600 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:38:08.894600 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:38:08.894600 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:38:08.894600 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:38:08.894600 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 12:38:08.894600 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:08.894600 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 189/388] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 26 12:38:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:38:08.901213 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad2e692-b4bd-4c69-9e81-c4e147124704 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:08.988118 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53beb8a7-e0bf-466d-8b8f-2c0712e7d33a None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:09.071505 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:09.076154 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e13930-ae2f-4fab-90c5-d5df67889fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:09.077470 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e13930-ae2f-4fab-90c5-d5df67889fe9 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:09.077749 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e13930-ae2f-4fab-90c5-d5df67889fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:09.078023 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e13930-ae2f-4fab-90c5-d5df67889fe9 None admin] PATH_INFO: `/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:09.080416 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad2e692-b4bd-4c69-9e81-c4e147124704 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:09.082651 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e13930-ae2f-4fab-90c5-d5df67889fe9 None admin] RBAC: Authorizing `identity:get_project(project_id=f5aedd2d5ee342aa80dfd96a062df0d9)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:09.085159 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e13930-ae2f-4fab-90c5-d5df67889fe9 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:09.088514 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 200/389] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:38:08 2022] GET /identity/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9 => generated 298 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:09.101336 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e13930-ae2f-4fab-90c5-d5df67889fe9 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:09.128308 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c6999-ce5c-4da6-acf3-72ad332d9a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:09.129202 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c6999-ce5c-4da6-acf3-72ad332d9a6d None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:09.129490 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c6999-ce5c-4da6-acf3-72ad332d9a6d None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:09.129760 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c6999-ce5c-4da6-acf3-72ad332d9a6d None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:09.130470 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17c6999-ce5c-4da6-acf3-72ad332d9a6d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:09.130768 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17c6999-ce5c-4da6-acf3-72ad332d9a6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:09.134446 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17c6999-ce5c-4da6-acf3-72ad332d9a6d None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:09.145427 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 190/390] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:38:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:09.152487 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c6999-ce5c-4da6-acf3-72ad332d9a6d None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:09.265684 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:09.275529 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e13930-ae2f-4fab-90c5-d5df67889fe9 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:09.310487 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c24e261-adcc-4ec5-95b2-66b377afd090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:09.311304 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c24e261-adcc-4ec5-95b2-66b377afd090 None admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:09.311712 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c24e261-adcc-4ec5-95b2-66b377afd090 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:09.312089 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c24e261-adcc-4ec5-95b2-66b377afd090 None admin] PATH_INFO: `/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9/groups/0869b90671e6441387d6cda07eeb9c4f/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:09.314104 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c24e261-adcc-4ec5-95b2-66b377afd090 None admin] RBAC: Authorizing `identity:create_grant(project_id=f5aedd2d5ee342aa80dfd96a062df0d9, group_id=0869b90671e6441387d6cda07eeb9c4f, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:09.315689 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c24e261-adcc-4ec5-95b2-66b377afd090 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:09.329985 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 201/391] 10.222.0.20 () {58 vars in 1442 bytes} [Tue Jul 26 12:38:09 2022] PUT /identity/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9/groups/0869b90671e6441387d6cda07eeb9c4f/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:38:09.392201 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c6999-ce5c-4da6-acf3-72ad332d9a6d None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:09.450411 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0923fc2c-279b-4df9-affa-3763b3618cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:09.451875 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0923fc2c-279b-4df9-affa-3763b3618cbc None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:09.452332 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0923fc2c-279b-4df9-affa-3763b3618cbc None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:09.452755 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0923fc2c-279b-4df9-affa-3763b3618cbc None admin] PATH_INFO: `/v3/groups/0869b90671e6441387d6cda07eeb9c4f` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:09.461268 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0923fc2c-279b-4df9-affa-3763b3618cbc None admin] RBAC: Authorizing `identity:get_group(group_id=0869b90671e6441387d6cda07eeb9c4f)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:09.462811 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0923fc2c-279b-4df9-affa-3763b3618cbc None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:09.464913 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 191/392] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Jul 26 12:38:09 2022] GET /identity/v3/groups/0869b90671e6441387d6cda07eeb9c4f => generated 226 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:09.470517 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c24e261-adcc-4ec5-95b2-66b377afd090 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:09.582283 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c24e261-adcc-4ec5-95b2-66b377afd090 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:09.617980 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f8d2f6-a136-42ae-9f55-18aa164c09cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:09.618632 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f8d2f6-a136-42ae-9f55-18aa164c09cd None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:09.618905 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f8d2f6-a136-42ae-9f55-18aa164c09cd None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:09.619177 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f8d2f6-a136-42ae-9f55-18aa164c09cd None admin] PATH_INFO: `/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:09.620361 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f8d2f6-a136-42ae-9f55-18aa164c09cd None admin] RBAC: Authorizing `identity:get_project(project_id=f5aedd2d5ee342aa80dfd96a062df0d9)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:09.621928 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f8d2f6-a136-42ae-9f55-18aa164c09cd None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:09.623923 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 202/393] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:38:09 2022] GET /identity/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:09.632601 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0923fc2c-279b-4df9-affa-3763b3618cbc None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:09.667629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:09.758128 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0923fc2c-279b-4df9-affa-3763b3618cbc None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:09.802785 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aaa790-6d6d-47f9-aabc-8d4479d6357c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:09.804482 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aaa790-6d6d-47f9-aabc-8d4479d6357c None admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:09.804964 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aaa790-6d6d-47f9-aabc-8d4479d6357c None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:09.805473 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aaa790-6d6d-47f9-aabc-8d4479d6357c None admin] PATH_INFO: `/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9/groups/0869b90671e6441387d6cda07eeb9c4f/roles/6589779ddd834447a1ed1921e1c834f6` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:09.807957 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aaa790-6d6d-47f9-aabc-8d4479d6357c None admin] RBAC: Authorizing `identity:create_grant(project_id=f5aedd2d5ee342aa80dfd96a062df0d9, group_id=0869b90671e6441387d6cda07eeb9c4f, role_id=6589779ddd834447a1ed1921e1c834f6)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:09.809628 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aaa790-6d6d-47f9-aabc-8d4479d6357c None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:09.825359 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 192/394] 10.222.0.20 () {58 vars in 1442 bytes} [Tue Jul 26 12:38:09 2022] PUT /identity/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9/groups/0869b90671e6441387d6cda07eeb9c4f/roles/6589779ddd834447a1ed1921e1c834f6 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:38:10.202258 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab588cf-16a5-40e8-ade9-b34629a94c94 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:10.202857 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab588cf-16a5-40e8-ade9-b34629a94c94 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:10.202857 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab588cf-16a5-40e8-ade9-b34629a94c94 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:10.204676 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 203/395] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:10.210407 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d1543-53c9-48f4-9217-2896a5d6a886 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:10.210999 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d1543-53c9-48f4-9217-2896a5d6a886 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:10.211466 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d1543-53c9-48f4-9217-2896a5d6a886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:10.236872 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-822d1543-53c9-48f4-9217-2896a5d6a886 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:10.238435 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-822d1543-53c9-48f4-9217-2896a5d6a886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:10.320944 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 193/396] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:38:10.354790 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab588cf-16a5-40e8-ade9-b34629a94c94 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:10.396417 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:10.530524 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab588cf-16a5-40e8-ade9-b34629a94c94 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:10.585299 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3be07-0627-4698-9ded-7d7a8d99615e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:10.585995 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3be07-0627-4698-9ded-7d7a8d99615e None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:10.586292 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3be07-0627-4698-9ded-7d7a8d99615e None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:10.586566 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3be07-0627-4698-9ded-7d7a8d99615e None admin] PATH_INFO: `/v3/roles/member` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:10.601331 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030e761-4833-430f-902a-f5621f5a5a7d None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:10.602436 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030e761-4833-430f-902a-f5621f5a5a7d None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:10.603021 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030e761-4833-430f-902a-f5621f5a5a7d None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:10.603269 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c3be07-0627-4698-9ded-7d7a8d99615e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:38:10.605172 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 194/397] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 12:38:10.609012 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-b9c3be07-0627-4698-9ded-7d7a8d99615e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 12:38:10.609012 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:10.609012 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:10.609012 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:10.609012 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:10.609012 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:10.609012 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:10.609012 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:10.609012 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:10.609012 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:10.609012 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:10.609012 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:10.609012 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:38:10.609012 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:38:10.609012 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:38:10.609012 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:38:10.609931 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:38:10.609931 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:38:10.609931 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:38:10.609931 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:38:10.609931 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:10.609931 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:10.609931 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:10.609931 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:10.609931 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:10.609931 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:10.609931 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:10.609931 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:10.609931 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:10.609931 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:10.609931 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:10.609931 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:10.610796 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:10.610796 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:10.610796 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:10.610796 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:10.610796 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:38:10.610796 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:38:10.610796 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:38:10.610796 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:38:10.610796 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:38:10.610796 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:38:10.610796 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 12:38:10.610796 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:10.611903 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 204/398] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:38:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:10.614287 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10e312-97d5-4c91-938a-f60e21c6fec7 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:10.614821 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10e312-97d5-4c91-938a-f60e21c6fec7 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:10.615307 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10e312-97d5-4c91-938a-f60e21c6fec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:10.616650 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3be07-0627-4698-9ded-7d7a8d99615e None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:10.651524 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-8c10e312-97d5-4c91-938a-f60e21c6fec7 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:10.652764 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-8c10e312-97d5-4c91-938a-f60e21c6fec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:10.685719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:10.713257 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:10.740995 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3be07-0627-4698-9ded-7d7a8d99615e None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:10.786256 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d82fc4f-21db-4c7b-8758-7dfebb994b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:10.786810 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82fc4f-21db-4c7b-8758-7dfebb994b00 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:10.787096 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82fc4f-21db-4c7b-8758-7dfebb994b00 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:10.787366 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82fc4f-21db-4c7b-8758-7dfebb994b00 None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:10.788715 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d82fc4f-21db-4c7b-8758-7dfebb994b00 None admin] RBAC: Adding query filter params (name=member) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:10.789023 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d82fc4f-21db-4c7b-8758-7dfebb994b00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:10.790439 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d82fc4f-21db-4c7b-8758-7dfebb994b00 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:10.811511 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 205/399] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 26 12:38:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:10.812938 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4f7ff-d70b-4949-9702-7d79e5d6d114 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:10.813224 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4f7ff-d70b-4949-9702-7d79e5d6d114 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:10.813505 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4f7ff-d70b-4949-9702-7d79e5d6d114 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:10.815199 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 206/400] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:10.818659 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd4f7ff-d70b-4949-9702-7d79e5d6d114 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:10.828217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 195/401] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:38:10.829962 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad27994-f017-439b-957d-8d6cc84d5c80 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:10.830459 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad27994-f017-439b-957d-8d6cc84d5c80 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:10.831590 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad27994-f017-439b-957d-8d6cc84d5c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:10.863304 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-0ad27994-f017-439b-957d-8d6cc84d5c80 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:10.868447 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-0ad27994-f017-439b-957d-8d6cc84d5c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:10.946020 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd4f7ff-d70b-4949-9702-7d79e5d6d114 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:10.986170 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7288bf42-4145-496f-8915-ced6b220113c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:10.987097 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7288bf42-4145-496f-8915-ced6b220113c None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:10.987624 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7288bf42-4145-496f-8915-ced6b220113c None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:10.987965 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7288bf42-4145-496f-8915-ced6b220113c None admin] PATH_INFO: `/v3/groups/0869b90671e6441387d6cda07eeb9c4f` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:10.989545 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7288bf42-4145-496f-8915-ced6b220113c None admin] RBAC: Authorizing `identity:get_group(group_id=0869b90671e6441387d6cda07eeb9c4f)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:10.991752 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7288bf42-4145-496f-8915-ced6b220113c None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:10.993960 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 207/402] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Jul 26 12:38:10 2022] GET /identity/v3/groups/0869b90671e6441387d6cda07eeb9c4f => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:10.994871 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7288bf42-4145-496f-8915-ced6b220113c None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:11.011282 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:11.024832 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 196/403] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:38:11.031685 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad27994-f017-439b-957d-8d6cc84d5c80 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:11.145024 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:11.173070 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad27994-f017-439b-957d-8d6cc84d5c80 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:11.173373 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7288bf42-4145-496f-8915-ced6b220113c None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:11.230075 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0193c2c0-72b8-45b0-83ac-7d699d4346ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:11.230816 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193c2c0-72b8-45b0-83ac-7d699d4346ee None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:11.231131 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193c2c0-72b8-45b0-83ac-7d699d4346ee None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:11.231507 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193c2c0-72b8-45b0-83ac-7d699d4346ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:11.253076 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0193c2c0-72b8-45b0-83ac-7d699d4346ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:38:11.260533 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-0193c2c0-72b8-45b0-83ac-7d699d4346ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:38:11.260533 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:11.260533 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:11.260533 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:11.260533 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:11.260533 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:11.260533 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:11.260533 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:11.260533 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:11.260533 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:11.260533 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:11.260533 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:11.260533 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:38:11.260533 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:38:11.260533 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:38:11.260533 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:38:11.261414 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:38:11.261414 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:38:11.261414 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:38:11.261414 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:38:11.261414 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:11.261414 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:11.261414 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:11.261414 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:11.261414 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:11.261414 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:11.261414 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:11.261414 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:11.261414 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:11.261414 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:11.261414 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:11.261414 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:11.262202 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:11.262202 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:11.262202 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:11.262202 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:11.262202 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:38:11.262202 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:38:11.262202 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:38:11.262202 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:38:11.262202 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:38:11.262202 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:38:11.262202 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:38:11.262202 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:11.263474 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0977520-f549-430a-8341-fe2aff5e1208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:11.264347 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0977520-f549-430a-8341-fe2aff5e1208 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:11.264767 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0977520-f549-430a-8341-fe2aff5e1208 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:11.265169 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0977520-f549-430a-8341-fe2aff5e1208 None admin] PATH_INFO: `/v3/projects/eb19926b01264c0a80580d9e18a4397d` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:11.266490 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0977520-f549-430a-8341-fe2aff5e1208 None admin] RBAC: Authorizing `identity:get_project(project_id=eb19926b01264c0a80580d9e18a4397d)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:11.268101 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0977520-f549-430a-8341-fe2aff5e1208 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:11.270213 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 197/404] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:38:11 2022] GET /identity/v3/projects/eb19926b01264c0a80580d9e18a4397d => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:11.271208 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0977520-f549-430a-8341-fe2aff5e1208 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:11.280667 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 208/405] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 26 12:38:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:11.281835 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0193c2c0-72b8-45b0-83ac-7d699d4346ee None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:11.418586 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:11.488867 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0193c2c0-72b8-45b0-83ac-7d699d4346ee None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:11.493627 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0977520-f549-430a-8341-fe2aff5e1208 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:11.565310 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5891b5-4b6f-45c4-a553-1bd5deb2a8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:11.567227 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5891b5-4b6f-45c4-a553-1bd5deb2a8fc None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:11.568046 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5891b5-4b6f-45c4-a553-1bd5deb2a8fc None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:11.569091 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5891b5-4b6f-45c4-a553-1bd5deb2a8fc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:11.569936 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c6966-c30e-4560-b878-fc6cccfea13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:11.571309 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c6966-c30e-4560-b878-fc6cccfea13c None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:11.571987 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c6966-c30e-4560-b878-fc6cccfea13c None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:11.573602 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c6966-c30e-4560-b878-fc6cccfea13c None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:11.574568 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63c6966-c30e-4560-b878-fc6cccfea13c None admin] RBAC: Adding query filter params (name=admin) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:11.576206 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63c6966-c30e-4560-b878-fc6cccfea13c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:11.578926 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63c6966-c30e-4560-b878-fc6cccfea13c None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:11.596203 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5891b5-4b6f-45c4-a553-1bd5deb2a8fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:38:11.598674 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 209/406] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:38:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:11.608357 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-6e5891b5-4b6f-45c4-a553-1bd5deb2a8fc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 12:38:11.608357 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:11.608357 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:11.608357 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:11.608357 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:11.608357 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:11.608357 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:11.608357 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:11.608357 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:11.608357 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:11.608357 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:11.608357 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:11.608357 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:38:11.608357 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:38:11.608357 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:38:11.608357 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:38:11.609590 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:38:11.609590 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:38:11.609590 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:38:11.609590 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:38:11.609590 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:11.609590 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:11.609590 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:11.609590 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:11.609590 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:11.609590 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:11.609590 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:11.609590 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:11.609590 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:11.609590 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:11.609590 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:11.609590 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:11.610353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:11.610353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:11.610353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:11.610353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:11.610353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:38:11.610353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:38:11.610353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:38:11.610353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:38:11.610353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:38:11.610353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:38:11.610353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 12:38:11.610353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:11.612519 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c6966-c30e-4560-b878-fc6cccfea13c None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:11.614777 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 198/407] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 26 12:38:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:11.621575 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5891b5-4b6f-45c4-a553-1bd5deb2a8fc None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:11.640537 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:11.750397 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5891b5-4b6f-45c4-a553-1bd5deb2a8fc None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:11.775423 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c6966-c30e-4560-b878-fc6cccfea13c None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:11.802707 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589aaae6-ca17-49e9-ba10-a026836ccd32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:11.803908 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589aaae6-ca17-49e9-ba10-a026836ccd32 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:11.804187 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589aaae6-ca17-49e9-ba10-a026836ccd32 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:11.804457 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589aaae6-ca17-49e9-ba10-a026836ccd32 None admin] PATH_INFO: `/v3/groups/c31a89a75973498f87919d406a571a79` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:11.805642 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589aaae6-ca17-49e9-ba10-a026836ccd32 None admin] RBAC: Authorizing `identity:get_group(group_id=c31a89a75973498f87919d406a571a79)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:11.807910 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589aaae6-ca17-49e9-ba10-a026836ccd32 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:11.811189 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 199/408] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Jul 26 12:38:11 2022] GET /identity/v3/groups/c31a89a75973498f87919d406a571a79 => generated 229 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:11.811992 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589aaae6-ca17-49e9-ba10-a026836ccd32 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:11.823790 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:11.830930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af00380-0d9b-432a-be65-25ae0723e9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:11.831598 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af00380-0d9b-432a-be65-25ae0723e9e7 None admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:11.831926 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af00380-0d9b-432a-be65-25ae0723e9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:11.832316 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af00380-0d9b-432a-be65-25ae0723e9e7 None admin] PATH_INFO: `/v3/projects/eb19926b01264c0a80580d9e18a4397d/groups/0869b90671e6441387d6cda07eeb9c4f/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:11.834246 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af00380-0d9b-432a-be65-25ae0723e9e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb19926b01264c0a80580d9e18a4397d, group_id=0869b90671e6441387d6cda07eeb9c4f, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:11.838905 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af00380-0d9b-432a-be65-25ae0723e9e7 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:11.863401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 210/409] 10.222.0.20 () {58 vars in 1442 bytes} [Tue Jul 26 12:38:11 2022] PUT /identity/v3/projects/eb19926b01264c0a80580d9e18a4397d/groups/0869b90671e6441387d6cda07eeb9c4f/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:38:11.863401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff43248b-b4ea-4ddc-bc63-adcddb6755c0 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:11.863401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff43248b-b4ea-4ddc-bc63-adcddb6755c0 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:11.863401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff43248b-b4ea-4ddc-bc63-adcddb6755c0 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:11.869311 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 211/410] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:11.869311 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff43248b-b4ea-4ddc-bc63-adcddb6755c0 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:11.945060 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:11.980821 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589aaae6-ca17-49e9-ba10-a026836ccd32 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:12.025600 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388ff47-8608-42df-babc-827ee1f99d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:12.026631 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388ff47-8608-42df-babc-827ee1f99d39 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:12.027128 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388ff47-8608-42df-babc-827ee1f99d39 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:12.027617 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388ff47-8608-42df-babc-827ee1f99d39 None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:12.028648 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e388ff47-8608-42df-babc-827ee1f99d39 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:12.029175 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e388ff47-8608-42df-babc-827ee1f99d39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:12.031132 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e388ff47-8608-42df-babc-827ee1f99d39 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:12.033528 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff43248b-b4ea-4ddc-bc63-adcddb6755c0 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:12.047492 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 200/411] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:38:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:12.049347 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d141f8-0a27-48d0-a037-69fa9b101819 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:12.049648 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d141f8-0a27-48d0-a037-69fa9b101819 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:12.049933 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d141f8-0a27-48d0-a037-69fa9b101819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:12.069770 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-c6d141f8-0a27-48d0-a037-69fa9b101819 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:12.071258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-c6d141f8-0a27-48d0-a037-69fa9b101819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:12.079975 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebae56-e9e8-45f4-9831-092b78cfc0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:12.080430 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebae56-e9e8-45f4-9831-092b78cfc0f2 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:12.080601 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebae56-e9e8-45f4-9831-092b78cfc0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:12.080880 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebae56-e9e8-45f4-9831-092b78cfc0f2 None admin] PATH_INFO: `/v3/projects/1ca671277c17458082e4383a242d5404` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:12.085020 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ebae56-e9e8-45f4-9831-092b78cfc0f2 None admin] RBAC: Authorizing `identity:get_project(project_id=1ca671277c17458082e4383a242d5404)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:12.086016 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ebae56-e9e8-45f4-9831-092b78cfc0f2 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:12.087993 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 212/412] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:38:11 2022] GET /identity/v3/projects/1ca671277c17458082e4383a242d5404 => generated 344 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:12.089293 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebae56-e9e8-45f4-9831-092b78cfc0f2 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:12.170310 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 201/413] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:38:12.171883 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d141f8-0a27-48d0-a037-69fa9b101819 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:12.180267 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:12.232361 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebae56-e9e8-45f4-9831-092b78cfc0f2 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:12.254977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:12.270414 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a64c125-23eb-43a4-a592-50768687b205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:12.271055 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a64c125-23eb-43a4-a592-50768687b205 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:12.271336 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a64c125-23eb-43a4-a592-50768687b205 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:12.271611 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a64c125-23eb-43a4-a592-50768687b205 None admin] PATH_INFO: `/v3/groups/0869b90671e6441387d6cda07eeb9c4f` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:12.272939 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a64c125-23eb-43a4-a592-50768687b205 None admin] RBAC: Authorizing `identity:get_group(group_id=0869b90671e6441387d6cda07eeb9c4f)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:12.274388 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a64c125-23eb-43a4-a592-50768687b205 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:12.276077 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 213/414] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Jul 26 12:38:12 2022] GET /identity/v3/groups/0869b90671e6441387d6cda07eeb9c4f => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:12.277153 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde27e3-4d1d-4bda-99e8-2482df0fdb4f None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:12.277438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde27e3-4d1d-4bda-99e8-2482df0fdb4f None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:12.277714 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde27e3-4d1d-4bda-99e8-2482df0fdb4f None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:12.282984 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 214/415] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:12.282984 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dde27e3-4d1d-4bda-99e8-2482df0fdb4f None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:12.303182 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d141f8-0a27-48d0-a037-69fa9b101819 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:12.340350 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2699c2c8-ea65-478f-92db-b2b90b157b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:12.341032 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699c2c8-ea65-478f-92db-b2b90b157b1f None admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:12.341317 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699c2c8-ea65-478f-92db-b2b90b157b1f None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:12.341585 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699c2c8-ea65-478f-92db-b2b90b157b1f None admin] PATH_INFO: `/v3/projects/1ca671277c17458082e4383a242d5404/groups/c31a89a75973498f87919d406a571a79/roles/0e78444e43c744668178971af4087ca0` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:12.343533 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2699c2c8-ea65-478f-92db-b2b90b157b1f None admin] RBAC: Authorizing `identity:create_grant(project_id=1ca671277c17458082e4383a242d5404, group_id=c31a89a75973498f87919d406a571a79, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:12.345036 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2699c2c8-ea65-478f-92db-b2b90b157b1f None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:12.371672 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 202/416] 10.222.0.20 () {58 vars in 1442 bytes} [Tue Jul 26 12:38:12 2022] PUT /identity/v3/projects/1ca671277c17458082e4383a242d5404/groups/c31a89a75973498f87919d406a571a79/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:38:12.371672 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2699c2c8-ea65-478f-92db-b2b90b157b1f None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:12.402244 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dde27e3-4d1d-4bda-99e8-2482df0fdb4f None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:12.443962 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:12.463461 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c82f329-3ba9-451c-bd4c-a8c34b68ee85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:12.464206 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82f329-3ba9-451c-bd4c-a8c34b68ee85 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:12.464537 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82f329-3ba9-451c-bd4c-a8c34b68ee85 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:12.464811 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82f329-3ba9-451c-bd4c-a8c34b68ee85 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:12.481349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c82f329-3ba9-451c-bd4c-a8c34b68ee85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:38:12.488672 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-5c82f329-3ba9-451c-bd4c-a8c34b68ee85 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 12:38:12.488672 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:12.488672 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:12.488672 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:12.488672 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:12.488672 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:12.488672 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:12.488672 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:12.488672 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:12.488672 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:12.488672 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:12.488672 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:12.488672 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:38:12.488672 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:38:12.488672 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:38:12.488672 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:38:12.489880 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:38:12.489880 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:38:12.489880 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:38:12.489880 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:38:12.489880 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:12.489880 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:12.489880 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:12.489880 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:12.489880 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:12.489880 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:12.489880 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:12.489880 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:12.489880 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:12.489880 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:12.489880 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:12.489880 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:12.490680 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:12.490680 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:12.490680 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:12.490680 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:12.490680 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:38:12.490680 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:38:12.490680 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:38:12.490680 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:38:12.490680 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:38:12.490680 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:38:12.490680 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 12:38:12.490680 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:12.490680 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 215/417] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:38:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:12.492431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891137ba-e1a7-44d4-82f7-0940059b1079 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:12.492587 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891137ba-e1a7-44d4-82f7-0940059b1079 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:12.492877 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891137ba-e1a7-44d4-82f7-0940059b1079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:12.511303 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2699c2c8-ea65-478f-92db-b2b90b157b1f None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:12.517180 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-891137ba-e1a7-44d4-82f7-0940059b1079 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:12.518456 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-891137ba-e1a7-44d4-82f7-0940059b1079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:12.545826 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:12.563797 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822e7cd7-eb3d-47e2-a634-066e419b77a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:12.564458 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e7cd7-eb3d-47e2-a634-066e419b77a9 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:12.564745 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e7cd7-eb3d-47e2-a634-066e419b77a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:12.565045 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e7cd7-eb3d-47e2-a634-066e419b77a9 None admin] PATH_INFO: `/v3/projects/eb19926b01264c0a80580d9e18a4397d` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:12.566245 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822e7cd7-eb3d-47e2-a634-066e419b77a9 None admin] RBAC: Authorizing `identity:get_project(project_id=eb19926b01264c0a80580d9e18a4397d)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:12.567956 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822e7cd7-eb3d-47e2-a634-066e419b77a9 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:12.570650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 203/418] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:38:12 2022] GET /identity/v3/projects/eb19926b01264c0a80580d9e18a4397d => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:12.571454 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822e7cd7-eb3d-47e2-a634-066e419b77a9 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:12.609656 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 216/419] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:38:12.610498 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891137ba-e1a7-44d4-82f7-0940059b1079 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:12.703753 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822e7cd7-eb3d-47e2-a634-066e419b77a9 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:12.720217 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891137ba-e1a7-44d4-82f7-0940059b1079 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:12.738612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:12.741648 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b99dd-58b3-4dc4-84a6-8fb298cbcdd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:12.742570 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b99dd-58b3-4dc4-84a6-8fb298cbcdd8 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:12.743053 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b99dd-58b3-4dc4-84a6-8fb298cbcdd8 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:12.743548 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b99dd-58b3-4dc4-84a6-8fb298cbcdd8 None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:12.744528 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4b99dd-58b3-4dc4-84a6-8fb298cbcdd8 None admin] RBAC: Adding query filter params (name=member) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:12.745052 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4b99dd-58b3-4dc4-84a6-8fb298cbcdd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:12.746670 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4b99dd-58b3-4dc4-84a6-8fb298cbcdd8 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:12.756969 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 204/420] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 26 12:38:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:12.758307 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b99dd-58b3-4dc4-84a6-8fb298cbcdd8 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:12.763542 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65db27c0-4d41-4c45-9368-f22798e3ff6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:12.764385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65db27c0-4d41-4c45-9368-f22798e3ff6c None admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:12.764667 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65db27c0-4d41-4c45-9368-f22798e3ff6c None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:12.764944 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65db27c0-4d41-4c45-9368-f22798e3ff6c None admin] PATH_INFO: `/v3/projects/eb19926b01264c0a80580d9e18a4397d/groups/0869b90671e6441387d6cda07eeb9c4f/roles/6589779ddd834447a1ed1921e1c834f6` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:12.767189 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65db27c0-4d41-4c45-9368-f22798e3ff6c None admin] RBAC: Authorizing `identity:create_grant(project_id=eb19926b01264c0a80580d9e18a4397d, group_id=0869b90671e6441387d6cda07eeb9c4f, role_id=6589779ddd834447a1ed1921e1c834f6)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:12.768743 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65db27c0-4d41-4c45-9368-f22798e3ff6c None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:12.785240 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 217/421] 10.222.0.20 () {58 vars in 1442 bytes} [Tue Jul 26 12:38:12 2022] PUT /identity/v3/projects/eb19926b01264c0a80580d9e18a4397d/groups/0869b90671e6441387d6cda07eeb9c4f/roles/6589779ddd834447a1ed1921e1c834f6 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:38:12.786812 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65db27c0-4d41-4c45-9368-f22798e3ff6c None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:12.843700 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:12.898897 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b99dd-58b3-4dc4-84a6-8fb298cbcdd8 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:12.933953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65db27c0-4d41-4c45-9368-f22798e3ff6c None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:12.950047 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5405df66-9021-41b0-8a65-5701e0c10c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:12.951082 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5405df66-9021-41b0-8a65-5701e0c10c17 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:12.951599 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5405df66-9021-41b0-8a65-5701e0c10c17 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:12.952109 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5405df66-9021-41b0-8a65-5701e0c10c17 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:12.973954 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18cf3cd-7029-4575-8698-47df61e3d9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:12.974592 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18cf3cd-7029-4575-8698-47df61e3d9c2 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:12.974876 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18cf3cd-7029-4575-8698-47df61e3d9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:12.975170 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18cf3cd-7029-4575-8698-47df61e3d9c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:12.988402 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5405df66-9021-41b0-8a65-5701e0c10c17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:38:12.995022 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-5405df66-9021-41b0-8a65-5701e0c10c17 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 12:38:12.995022 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:12.995022 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:12.995022 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:12.995022 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:12.995022 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:12.995022 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:12.995022 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:12.995022 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:12.995022 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:12.995022 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:12.995022 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:12.995022 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:38:12.995022 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:38:12.995022 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:38:12.995022 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:38:13.000330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:38:13.000330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:38:13.000330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:38:13.000330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:38:13.000330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:13.000330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:13.000330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:13.000330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:13.000330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:13.000330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:13.000330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:13.000330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:13.000330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:13.000330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:13.000330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:13.000330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:13.001141 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:13.001141 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:13.001141 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:13.001141 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:13.001141 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:38:13.001141 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:38:13.001141 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:38:13.001141 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:38:13.001141 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:38:13.001141 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:38:13.001141 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 12:38:13.001141 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:13.003568 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 205/422] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 26 12:38:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:13.011384 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5405df66-9021-41b0-8a65-5701e0c10c17 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:13.064543 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18cf3cd-7029-4575-8698-47df61e3d9c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:13.066317 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18cf3cd-7029-4575-8698-47df61e3d9c2 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:13.072053 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:13.133633 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5405df66-9021-41b0-8a65-5701e0c10c17 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:13.153864 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:13.176904 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b207b2aa-a241-434a-971d-76d0dea52fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:13.178026 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207b2aa-a241-434a-971d-76d0dea52fa3 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:13.179042 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207b2aa-a241-434a-971d-76d0dea52fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:13.179786 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207b2aa-a241-434a-971d-76d0dea52fa3 None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:13.180720 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b207b2aa-a241-434a-971d-76d0dea52fa3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:13.181743 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b207b2aa-a241-434a-971d-76d0dea52fa3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:13.183525 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b207b2aa-a241-434a-971d-76d0dea52fa3 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:13.193259 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 206/423] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:38:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:13.205302 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 218/424] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:38:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 12:38:13.206690 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b207b2aa-a241-434a-971d-76d0dea52fa3 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:13.211254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18cf3cd-7029-4575-8698-47df61e3d9c2 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:13.373853 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:13.385451 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b207b2aa-a241-434a-971d-76d0dea52fa3 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:13.424738 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37088ea6-53fd-4bd2-9362-b737dbf5c119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:13.426425 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37088ea6-53fd-4bd2-9362-b737dbf5c119 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:13.426919 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37088ea6-53fd-4bd2-9362-b737dbf5c119 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:13.427659 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37088ea6-53fd-4bd2-9362-b737dbf5c119 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:13.442811 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18cf3cd-7029-4575-8698-47df61e3d9c2 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:13.487132 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:13.514234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37088ea6-53fd-4bd2-9362-b737dbf5c119 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:13.516095 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37088ea6-53fd-4bd2-9362-b737dbf5c119 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:13.545332 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952383c8-cc1a-4506-8d78-85060caafce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:13.546010 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952383c8-cc1a-4506-8d78-85060caafce3 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:13.546296 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952383c8-cc1a-4506-8d78-85060caafce3 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:13.546574 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952383c8-cc1a-4506-8d78-85060caafce3 None admin] PATH_INFO: `/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:13.552722 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952383c8-cc1a-4506-8d78-85060caafce3 None admin] RBAC: Authorizing `identity:get_project(project_id=f5aedd2d5ee342aa80dfd96a062df0d9)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:13.554391 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952383c8-cc1a-4506-8d78-85060caafce3 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:13.556531 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 219/425] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:38:13 2022] GET /identity/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9 => generated 298 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:13.561852 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952383c8-cc1a-4506-8d78-85060caafce3 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:13.635133 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 207/426] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:38:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 12:38:13.641536 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37088ea6-53fd-4bd2-9362-b737dbf5c119 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:13.736010 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:13.789679 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37088ea6-53fd-4bd2-9362-b737dbf5c119 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:13.800451 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952383c8-cc1a-4506-8d78-85060caafce3 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:13.838360 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ba300d-8a0c-4e6b-8bf6-bfcdbcf9164c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:13.838999 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba300d-8a0c-4e6b-8bf6-bfcdbcf9164c None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:13.839280 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba300d-8a0c-4e6b-8bf6-bfcdbcf9164c None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:13.839551 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba300d-8a0c-4e6b-8bf6-bfcdbcf9164c None admin] PATH_INFO: `/v3/groups/0869b90671e6441387d6cda07eeb9c4f` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:13.840862 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ba300d-8a0c-4e6b-8bf6-bfcdbcf9164c None admin] RBAC: Authorizing `identity:get_group(group_id=0869b90671e6441387d6cda07eeb9c4f)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:13.842488 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ba300d-8a0c-4e6b-8bf6-bfcdbcf9164c None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:13.844457 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 220/427] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Jul 26 12:38:13 2022] GET /identity/v3/groups/0869b90671e6441387d6cda07eeb9c4f => generated 226 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:38:13.856557 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ba300d-8a0c-4e6b-8bf6-bfcdbcf9164c None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:13.881279 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd72a750-ff86-4f57-9b8a-ec73f51641b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:13.881990 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72a750-ff86-4f57-9b8a-ec73f51641b5 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:13.882270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72a750-ff86-4f57-9b8a-ec73f51641b5 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:13.882545 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72a750-ff86-4f57-9b8a-ec73f51641b5 None admin] PATH_INFO: `/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:13.883749 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd72a750-ff86-4f57-9b8a-ec73f51641b5 None admin] RBAC: Authorizing `identity:get_project(project_id=f5aedd2d5ee342aa80dfd96a062df0d9)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:13.885210 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd72a750-ff86-4f57-9b8a-ec73f51641b5 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:13.887176 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 208/428] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:38:13 2022] GET /identity/v3/projects/f5aedd2d5ee342aa80dfd96a062df0d9 => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:38:13.900067 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:13.900814 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd72a750-ff86-4f57-9b8a-ec73f51641b5 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:13.974433 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ba300d-8a0c-4e6b-8bf6-bfcdbcf9164c None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:14.052380 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e0eded-2fb3-44c6-8ed3-df04d90e0074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:14.053924 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0eded-2fb3-44c6-8ed3-df04d90e0074 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:14.055491 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0eded-2fb3-44c6-8ed3-df04d90e0074 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:14.056008 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0eded-2fb3-44c6-8ed3-df04d90e0074 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:14.057005 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e0eded-2fb3-44c6-8ed3-df04d90e0074 None admin] RBAC: Adding query filter params (group.id=0869b90671e6441387d6cda07eeb9c4f, role.id=01c9179cca2747a6b95b0decfca8b7ae, scope.project.id=f5aedd2d5ee342aa80dfd96a062df0d9) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:14.057744 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e0eded-2fb3-44c6-8ed3-df04d90e0074 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:14.060139 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e0eded-2fb3-44c6-8ed3-df04d90e0074 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:14.075800 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 221/429] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Jul 26 12:38:13 2022] GET /identity/v3/role_assignments?group.id=0869b90671e6441387d6cda07eeb9c4f&scope.project.id=f5aedd2d5ee342aa80dfd96a062df0d9&role.id=01c9179cca2747a6b95b0decfca8b7ae => generated 611 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:14.099652 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd72a750-ff86-4f57-9b8a-ec73f51641b5 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:14.157994 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607104c7-5843-4ac9-94c3-0dd079f3b6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:14.158575 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607104c7-5843-4ac9-94c3-0dd079f3b6d7 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:14.158852 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607104c7-5843-4ac9-94c3-0dd079f3b6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:14.159124 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607104c7-5843-4ac9-94c3-0dd079f3b6d7 None admin] PATH_INFO: `/v3/groups/0869b90671e6441387d6cda07eeb9c4f` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:14.164590 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607104c7-5843-4ac9-94c3-0dd079f3b6d7 None admin] RBAC: Authorizing `identity:get_group(group_id=0869b90671e6441387d6cda07eeb9c4f)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:14.171741 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607104c7-5843-4ac9-94c3-0dd079f3b6d7 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:14.171741 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 209/430] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Jul 26 12:38:13 2022] GET /identity/v3/groups/0869b90671e6441387d6cda07eeb9c4f => generated 226 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:14.173906 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e0eded-2fb3-44c6-8ed3-df04d90e0074 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:14.280036 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:14.310138 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e0eded-2fb3-44c6-8ed3-df04d90e0074 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:14.352822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f757cdde-4015-4eb5-bfae-ce8ad7b307da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:14.353808 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757cdde-4015-4eb5-bfae-ce8ad7b307da None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:14.354321 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757cdde-4015-4eb5-bfae-ce8ad7b307da None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:14.354798 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757cdde-4015-4eb5-bfae-ce8ad7b307da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:14.355817 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f757cdde-4015-4eb5-bfae-ce8ad7b307da None admin] RBAC: Adding query filter params (group.id=0869b90671e6441387d6cda07eeb9c4f, role.id=6589779ddd834447a1ed1921e1c834f6, scope.project.id=f5aedd2d5ee342aa80dfd96a062df0d9) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:14.356350 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f757cdde-4015-4eb5-bfae-ce8ad7b307da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:14.358016 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f757cdde-4015-4eb5-bfae-ce8ad7b307da None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:14.365156 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 222/431] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Jul 26 12:38:14 2022] GET /identity/v3/role_assignments?group.id=0869b90671e6441387d6cda07eeb9c4f&scope.project.id=f5aedd2d5ee342aa80dfd96a062df0d9&role.id=6589779ddd834447a1ed1921e1c834f6 => generated 611 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:14.382986 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853ce5c-e9f4-410d-b83e-5dddfff1ebc0 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:14.383321 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853ce5c-e9f4-410d-b83e-5dddfff1ebc0 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:14.383575 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853ce5c-e9f4-410d-b83e-5dddfff1ebc0 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:14.385003 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 210/432] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:14.391279 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead5ecc-b4d3-4bf5-834e-b70b0aedc2d2 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:14.392279 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead5ecc-b4d3-4bf5-834e-b70b0aedc2d2 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:14.392779 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead5ecc-b4d3-4bf5-834e-b70b0aedc2d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:14.425066 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-7ead5ecc-b4d3-4bf5-834e-b70b0aedc2d2 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:14.426215 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-7ead5ecc-b4d3-4bf5-834e-b70b0aedc2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:14.520032 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 223/433] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:38:14.544267 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8853ce5c-e9f4-410d-b83e-5dddfff1ebc0 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:14.567320 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:14.588785 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d0631-e67a-4f7e-bea2-2dba90e738ff None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:14.589144 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d0631-e67a-4f7e-bea2-2dba90e738ff None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:14.589338 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d0631-e67a-4f7e-bea2-2dba90e738ff None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:14.590697 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 224/434] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:14.595887 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec7913-0881-4728-8acc-1dde18b7248e None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:14.596171 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec7913-0881-4728-8acc-1dde18b7248e None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:14.596447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec7913-0881-4728-8acc-1dde18b7248e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:14.617272 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-bcec7913-0881-4728-8acc-1dde18b7248e None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:14.618450 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-bcec7913-0881-4728-8acc-1dde18b7248e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:14.676472 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8853ce5c-e9f4-410d-b83e-5dddfff1ebc0 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:14.703287 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:14.715732 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 225/435] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 12:38:14.722767 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13faea57-3cad-444a-8ed5-7cfa61692f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:14.722997 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13faea57-3cad-444a-8ed5-7cfa61692f30 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:14.723247 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13faea57-3cad-444a-8ed5-7cfa61692f30 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:14.723521 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13faea57-3cad-444a-8ed5-7cfa61692f30 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:14.734404 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcec7913-0881-4728-8acc-1dde18b7248e None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:14.737236 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13faea57-3cad-444a-8ed5-7cfa61692f30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:38:14.743090 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-13faea57-3cad-444a-8ed5-7cfa61692f30 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 12:38:14.743090 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:14.743090 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:14.743090 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:14.743090 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:14.743090 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:14.743090 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:14.743090 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:14.743090 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:14.743090 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:14.743090 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:14.743090 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:14.743090 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:38:14.743090 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:38:14.743090 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:38:14.743090 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:38:14.744206 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:38:14.744206 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:38:14.744206 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:38:14.744206 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:38:14.744206 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:14.744206 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:14.744206 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:14.744206 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:14.744206 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:14.744206 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:14.744206 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:14.744206 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:14.744206 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:14.744206 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:14.744206 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:14.744206 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:14.745192 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:14.745192 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:14.745192 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:14.745192 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:14.745192 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:38:14.745192 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:38:14.745192 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:38:14.745192 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:38:14.745192 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:38:14.745192 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:38:14.745192 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 12:38:14.745192 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:14.745192 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 211/436] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:38:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:14.749980 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13faea57-3cad-444a-8ed5-7cfa61692f30 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:14.860048 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:14.884890 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcec7913-0881-4728-8acc-1dde18b7248e None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:14.901924 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13faea57-3cad-444a-8ed5-7cfa61692f30 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:14.933151 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972b6522-459f-435c-bcca-50409256495e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:14.933732 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b6522-459f-435c-bcca-50409256495e None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:14.934012 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b6522-459f-435c-bcca-50409256495e None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:14.934284 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b6522-459f-435c-bcca-50409256495e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:14.953437 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972b6522-459f-435c-bcca-50409256495e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:38:14.958682 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-972b6522-459f-435c-bcca-50409256495e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:38:14.958682 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:14.958682 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:14.958682 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:14.958682 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:14.958682 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:14.958682 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:14.958682 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:14.958682 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:14.958682 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:14.958682 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:14.958682 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:14.958682 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:38:14.958682 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:38:14.958682 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:38:14.958682 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:38:14.959593 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:38:14.959593 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:38:14.959593 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:38:14.959593 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:38:14.959593 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:14.959593 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:14.959593 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:14.959593 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:14.959593 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:14.959593 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:14.959593 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:14.959593 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:14.959593 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:14.959593 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:14.959593 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:14.959593 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:14.960422 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:14.960422 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:14.960422 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:14.960422 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:14.960422 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:38:14.960422 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:38:14.960422 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:38:14.960422 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:38:14.960422 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:38:14.960422 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:38:14.960422 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:38:14.960422 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:14.961020 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f9cdb-b078-4b9b-b2cc-760319cf6676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:14.961120 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 226/437] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 26 12:38:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:38:14.961226 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f9cdb-b078-4b9b-b2cc-760319cf6676 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:14.961537 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f9cdb-b078-4b9b-b2cc-760319cf6676 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:14.961813 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f9cdb-b078-4b9b-b2cc-760319cf6676 None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:14.962754 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591f9cdb-b078-4b9b-b2cc-760319cf6676 None admin] RBAC: Adding query filter params (name=member) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:14.962858 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591f9cdb-b078-4b9b-b2cc-760319cf6676 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:14.964240 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591f9cdb-b078-4b9b-b2cc-760319cf6676 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:14.973733 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 212/438] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 26 12:38:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:38:14.974607 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972b6522-459f-435c-bcca-50409256495e None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:14.979970 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f9cdb-b078-4b9b-b2cc-760319cf6676 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:15.049380 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:15.102735 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972b6522-459f-435c-bcca-50409256495e None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:15.109534 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f9cdb-b078-4b9b-b2cc-760319cf6676 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:15.147810 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee34280-7966-49c7-a309-10b221d196cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:15.148546 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee34280-7966-49c7-a309-10b221d196cc None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:15.148834 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee34280-7966-49c7-a309-10b221d196cc None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:15.149105 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee34280-7966-49c7-a309-10b221d196cc None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:15.149824 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee34280-7966-49c7-a309-10b221d196cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:15.150127 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee34280-7966-49c7-a309-10b221d196cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:15.151438 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c7b63a-2c2d-430d-8d11-fccfa772c656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:15.151551 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee34280-7966-49c7-a309-10b221d196cc None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:15.152108 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7b63a-2c2d-430d-8d11-fccfa772c656 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:15.152385 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7b63a-2c2d-430d-8d11-fccfa772c656 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:15.152651 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7b63a-2c2d-430d-8d11-fccfa772c656 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:15.157391 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:15.166306 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 227/439] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:38:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:38:15.167210 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee34280-7966-49c7-a309-10b221d196cc None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:15.233250 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c7b63a-2c2d-430d-8d11-fccfa772c656 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:15.234759 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c7b63a-2c2d-430d-8d11-fccfa772c656 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:15.272635 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee34280-7966-49c7-a309-10b221d196cc None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:15.295087 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:15.308593 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a391744-b87c-4a6f-a122-ec96a3016a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:15.309209 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a391744-b87c-4a6f-a122-ec96a3016a66 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:15.309491 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a391744-b87c-4a6f-a122-ec96a3016a66 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:15.309767 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a391744-b87c-4a6f-a122-ec96a3016a66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:15.336305 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 213/440] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:38:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 12:38:15.337918 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b075c3a-2397-4f5f-a30e-0ec4e65a5977 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:15.338209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b075c3a-2397-4f5f-a30e-0ec4e65a5977 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:15.338481 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b075c3a-2397-4f5f-a30e-0ec4e65a5977 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:15.339747 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 214/441] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:15.341769 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b075c3a-2397-4f5f-a30e-0ec4e65a5977 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:15.376446 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:15.384051 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a391744-b87c-4a6f-a122-ec96a3016a66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:15.385516 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a391744-b87c-4a6f-a122-ec96a3016a66 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:15.442908 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b075c3a-2397-4f5f-a30e-0ec4e65a5977 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:15.476327 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6388b394-6db4-4b1f-a37b-f2bdd8ce53ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:15.477036 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388b394-6db4-4b1f-a37b-f2bdd8ce53ee None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:15.477317 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388b394-6db4-4b1f-a37b-f2bdd8ce53ee None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:15.477595 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388b394-6db4-4b1f-a37b-f2bdd8ce53ee None admin] PATH_INFO: `/v3/projects/eb19926b01264c0a80580d9e18a4397d` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:15.478741 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6388b394-6db4-4b1f-a37b-f2bdd8ce53ee None admin] RBAC: Authorizing `identity:get_project(project_id=eb19926b01264c0a80580d9e18a4397d)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:15.480214 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6388b394-6db4-4b1f-a37b-f2bdd8ce53ee None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:15.481629 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 228/442] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:38:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 12:38:15.482123 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 215/443] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:38:15 2022] GET /identity/v3/projects/eb19926b01264c0a80580d9e18a4397d => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:38:15.485034 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad16502-e19e-4f20-863e-1266dde6d2fe None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:15.485267 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad16502-e19e-4f20-863e-1266dde6d2fe None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:15.485539 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad16502-e19e-4f20-863e-1266dde6d2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:15.487064 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6388b394-6db4-4b1f-a37b-f2bdd8ce53ee None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:15.503539 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-6ad16502-e19e-4f20-863e-1266dde6d2fe None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:15.504768 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-6ad16502-e19e-4f20-863e-1266dde6d2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:15.520920 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:15.572865 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 229/444] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:38:15.587860 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad16502-e19e-4f20-863e-1266dde6d2fe None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:15.600146 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6388b394-6db4-4b1f-a37b-f2bdd8ce53ee None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:15.603082 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:15.635721 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab77fb-7183-4566-b34d-5ca4a7a6b261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:15.636582 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab77fb-7183-4566-b34d-5ca4a7a6b261 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:15.636858 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab77fb-7183-4566-b34d-5ca4a7a6b261 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:15.637129 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab77fb-7183-4566-b34d-5ca4a7a6b261 None admin] PATH_INFO: `/v3/projects/1ca671277c17458082e4383a242d5404` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:15.638282 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ab77fb-7183-4566-b34d-5ca4a7a6b261 None admin] RBAC: Authorizing `identity:get_project(project_id=1ca671277c17458082e4383a242d5404)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:15.639754 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ab77fb-7183-4566-b34d-5ca4a7a6b261 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:15.641427 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 216/445] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:38:15 2022] GET /identity/v3/projects/1ca671277c17458082e4383a242d5404 => generated 344 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:38:15.642182 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab77fb-7183-4566-b34d-5ca4a7a6b261 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:15.673002 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad16502-e19e-4f20-863e-1266dde6d2fe None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:15.711136 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d18cf4-8fcc-4ee6-8947-7c1b209384c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:15.711964 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d18cf4-8fcc-4ee6-8947-7c1b209384c7 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:15.712308 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d18cf4-8fcc-4ee6-8947-7c1b209384c7 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:15.712650 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d18cf4-8fcc-4ee6-8947-7c1b209384c7 None admin] PATH_INFO: `/v3/groups/0869b90671e6441387d6cda07eeb9c4f` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:15.713963 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d18cf4-8fcc-4ee6-8947-7c1b209384c7 None admin] RBAC: Authorizing `identity:get_group(group_id=0869b90671e6441387d6cda07eeb9c4f)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:15.715507 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d18cf4-8fcc-4ee6-8947-7c1b209384c7 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:15.717211 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 230/446] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Jul 26 12:38:15 2022] GET /identity/v3/groups/0869b90671e6441387d6cda07eeb9c4f => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:15.717960 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d18cf4-8fcc-4ee6-8947-7c1b209384c7 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:15.738975 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:15.751543 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab77fb-7183-4566-b34d-5ca4a7a6b261 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:15.785850 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d2ffc8-c87d-4762-8890-1a305bc8a77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:15.786492 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2ffc8-c87d-4762-8890-1a305bc8a77c None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:15.786772 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2ffc8-c87d-4762-8890-1a305bc8a77c None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:15.787057 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2ffc8-c87d-4762-8890-1a305bc8a77c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:15.797276 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d2ffc8-c87d-4762-8890-1a305bc8a77c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:38:15.802196 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-76d2ffc8-c87d-4762-8890-1a305bc8a77c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 12:38:15.802196 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:15.802196 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:15.802196 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:15.802196 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:15.802196 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:15.802196 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:15.802196 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:15.802196 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:15.802196 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:15.802196 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:15.802196 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:15.802196 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:38:15.802196 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:38:15.802196 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:38:15.802196 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:38:15.803241 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:38:15.803241 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:38:15.803241 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:38:15.803241 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:38:15.803241 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:15.803241 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:15.803241 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:15.803241 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:15.803241 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:15.803241 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:15.803241 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:15.803241 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:15.803241 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:15.803241 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:15.803241 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:15.803241 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:15.804265 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:15.804265 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:15.804265 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:15.804265 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:15.804265 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:38:15.804265 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:38:15.804265 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:38:15.804265 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:38:15.804265 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:38:15.804265 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:38:15.804265 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 12:38:15.804265 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:15.804265 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 217/447] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 26 12:38:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:15.804265 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d2ffc8-c87d-4762-8890-1a305bc8a77c None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:15.821649 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d18cf4-8fcc-4ee6-8947-7c1b209384c7 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:15.840434 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:15.856836 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23523fb7-c825-4e68-8dec-4ac18461a7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:15.857541 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23523fb7-c825-4e68-8dec-4ac18461a7fb None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:15.857752 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23523fb7-c825-4e68-8dec-4ac18461a7fb None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:15.858011 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23523fb7-c825-4e68-8dec-4ac18461a7fb None admin] PATH_INFO: `/v3/groups/c31a89a75973498f87919d406a571a79` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:15.859160 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23523fb7-c825-4e68-8dec-4ac18461a7fb None admin] RBAC: Authorizing `identity:get_group(group_id=c31a89a75973498f87919d406a571a79)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:15.860632 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23523fb7-c825-4e68-8dec-4ac18461a7fb None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:15.862241 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 231/448] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Jul 26 12:38:15 2022] GET /identity/v3/groups/c31a89a75973498f87919d406a571a79 => generated 229 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:15.862989 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23523fb7-c825-4e68-8dec-4ac18461a7fb None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:15.911139 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d2ffc8-c87d-4762-8890-1a305bc8a77c None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:15.945037 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043dd4b-09aa-4ac3-9c07-e3c2bbd6953b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:15.945642 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043dd4b-09aa-4ac3-9c07-e3c2bbd6953b None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:15.945918 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043dd4b-09aa-4ac3-9c07-e3c2bbd6953b None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:15.946187 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043dd4b-09aa-4ac3-9c07-e3c2bbd6953b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:15.946954 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7043dd4b-09aa-4ac3-9c07-e3c2bbd6953b None admin] RBAC: Adding query filter params (group.id=0869b90671e6441387d6cda07eeb9c4f, role.id=01c9179cca2747a6b95b0decfca8b7ae, scope.project.id=eb19926b01264c0a80580d9e18a4397d) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:15.947259 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7043dd4b-09aa-4ac3-9c07-e3c2bbd6953b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:15.948724 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7043dd4b-09aa-4ac3-9c07-e3c2bbd6953b None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:15.950957 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:15.955133 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 218/449] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Jul 26 12:38:15 2022] GET /identity/v3/role_assignments?group.id=0869b90671e6441387d6cda07eeb9c4f&scope.project.id=eb19926b01264c0a80580d9e18a4397d&role.id=01c9179cca2747a6b95b0decfca8b7ae => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:15.956149 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043dd4b-09aa-4ac3-9c07-e3c2bbd6953b None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:15.963767 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23523fb7-c825-4e68-8dec-4ac18461a7fb None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:16.008730 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a49c8d-0976-44ce-a8c6-e2717a8b1457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:16.009276 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a49c8d-0976-44ce-a8c6-e2717a8b1457 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:16.009558 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a49c8d-0976-44ce-a8c6-e2717a8b1457 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:16.009832 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a49c8d-0976-44ce-a8c6-e2717a8b1457 None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:16.010581 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a49c8d-0976-44ce-a8c6-e2717a8b1457 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:16.010941 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a49c8d-0976-44ce-a8c6-e2717a8b1457 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:16.012487 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a49c8d-0976-44ce-a8c6-e2717a8b1457 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:16.022376 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 232/450] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:38:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:16.028128 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a49c8d-0976-44ce-a8c6-e2717a8b1457 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:16.067567 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:16.074739 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043dd4b-09aa-4ac3-9c07-e3c2bbd6953b None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:16.114394 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d04fd87-c811-42ea-9491-81f38b6f75c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:16.115076 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d04fd87-c811-42ea-9491-81f38b6f75c6 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:16.115374 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d04fd87-c811-42ea-9491-81f38b6f75c6 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:16.115660 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d04fd87-c811-42ea-9491-81f38b6f75c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:16.116531 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d04fd87-c811-42ea-9491-81f38b6f75c6 None admin] RBAC: Adding query filter params (group.id=c31a89a75973498f87919d406a571a79, role.id=0e78444e43c744668178971af4087ca0, scope.project.id=1ca671277c17458082e4383a242d5404) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:16.116848 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d04fd87-c811-42ea-9491-81f38b6f75c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:16.118300 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d04fd87-c811-42ea-9491-81f38b6f75c6 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:16.124525 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 219/451] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Jul 26 12:38:15 2022] GET /identity/v3/role_assignments?group.id=c31a89a75973498f87919d406a571a79&scope.project.id=1ca671277c17458082e4383a242d5404&role.id=0e78444e43c744668178971af4087ca0 => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:16.142343 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a49c8d-0976-44ce-a8c6-e2717a8b1457 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:16.192507 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9639d93-9bec-47b6-9281-1e3921b8889b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:16.193144 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9639d93-9bec-47b6-9281-1e3921b8889b None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:16.193421 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9639d93-9bec-47b6-9281-1e3921b8889b None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:16.193691 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9639d93-9bec-47b6-9281-1e3921b8889b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:16.216687 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:16.279128 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9639d93-9bec-47b6-9281-1e3921b8889b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:16.280564 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9639d93-9bec-47b6-9281-1e3921b8889b None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:16.383422 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 233/452] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:38:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 12:38:16.389288 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d04fd87-c811-42ea-9491-81f38b6f75c6 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:16.487112 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d04fd87-c811-42ea-9491-81f38b6f75c6 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:16.519144 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e4447-a789-468b-be73-3db5a58fe211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:16.519792 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e4447-a789-468b-be73-3db5a58fe211 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:16.520071 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e4447-a789-468b-be73-3db5a58fe211 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:16.520340 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e4447-a789-468b-be73-3db5a58fe211 None admin] PATH_INFO: `/v3/projects/eb19926b01264c0a80580d9e18a4397d` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:16.521469 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11e4447-a789-468b-be73-3db5a58fe211 None admin] RBAC: Authorizing `identity:get_project(project_id=eb19926b01264c0a80580d9e18a4397d)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:16.522919 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11e4447-a789-468b-be73-3db5a58fe211 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:16.524575 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 220/453] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:38:16 2022] GET /identity/v3/projects/eb19926b01264c0a80580d9e18a4397d => generated 302 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:16.530282 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9639d93-9bec-47b6-9281-1e3921b8889b None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:16.577868 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:16.626139 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9639d93-9bec-47b6-9281-1e3921b8889b None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:16.657924 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcedaff-525e-4236-9315-641dd26e1f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:16.658544 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcedaff-525e-4236-9315-641dd26e1f8f None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:16.658821 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcedaff-525e-4236-9315-641dd26e1f8f None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:16.659091 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcedaff-525e-4236-9315-641dd26e1f8f None admin] PATH_INFO: `/v3/groups/0869b90671e6441387d6cda07eeb9c4f` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:16.660225 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcedaff-525e-4236-9315-641dd26e1f8f None admin] RBAC: Authorizing `identity:get_group(group_id=0869b90671e6441387d6cda07eeb9c4f)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:16.661657 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcedaff-525e-4236-9315-641dd26e1f8f None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:16.663776 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 234/454] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Jul 26 12:38:16 2022] GET /identity/v3/groups/0869b90671e6441387d6cda07eeb9c4f => generated 226 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:16.669616 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e4447-a789-468b-be73-3db5a58fe211 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:16.698149 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:16.768789 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e4447-a789-468b-be73-3db5a58fe211 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:16.810745 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e86af9-f585-43fc-8da7-4a337f423a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:16.811280 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e86af9-f585-43fc-8da7-4a337f423a79 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:16.811559 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e86af9-f585-43fc-8da7-4a337f423a79 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:16.811857 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e86af9-f585-43fc-8da7-4a337f423a79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:16.812868 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e86af9-f585-43fc-8da7-4a337f423a79 None admin] RBAC: Adding query filter params (group.id=0869b90671e6441387d6cda07eeb9c4f, role.id=6589779ddd834447a1ed1921e1c834f6, scope.project.id=eb19926b01264c0a80580d9e18a4397d) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:16.813180 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e86af9-f585-43fc-8da7-4a337f423a79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:16.814620 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e86af9-f585-43fc-8da7-4a337f423a79 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:16.820804 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 221/455] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Jul 26 12:38:16 2022] GET /identity/v3/role_assignments?group.id=0869b90671e6441387d6cda07eeb9c4f&scope.project.id=eb19926b01264c0a80580d9e18a4397d&role.id=6589779ddd834447a1ed1921e1c834f6 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:38:19.551811 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8f18c9-b911-488e-8610-863a6e363478 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:19.552334 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8f18c9-b911-488e-8610-863a6e363478 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:19.552334 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8f18c9-b911-488e-8610-863a6e363478 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:19.554009 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 235/456] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:19.559406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab59bda-6dd5-4b00-a2f4-53e11abb1e9c None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:19.559870 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab59bda-6dd5-4b00-a2f4-53e11abb1e9c None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:19.560044 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab59bda-6dd5-4b00-a2f4-53e11abb1e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:19.584706 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-dab59bda-6dd5-4b00-a2f4-53e11abb1e9c None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:19.586249 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-dab59bda-6dd5-4b00-a2f4-53e11abb1e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:19.656751 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09bb3a0-c6e5-43d6-8e8f-0fa1d8b648da None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:19.657090 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09bb3a0-c6e5-43d6-8e8f-0fa1d8b648da None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:19.657335 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09bb3a0-c6e5-43d6-8e8f-0fa1d8b648da None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:19.659112 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 236/457] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:19.673562 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 222/458] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 12:38:19.676077 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175a2ce-f04f-43f9-9e91-176563a8e381 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:19.676363 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175a2ce-f04f-43f9-9e91-176563a8e381 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:19.676639 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175a2ce-f04f-43f9-9e91-176563a8e381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:19.708614 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-5175a2ce-f04f-43f9-9e91-176563a8e381 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:19.709970 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-5175a2ce-f04f-43f9-9e91-176563a8e381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:19.714398 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab59bda-6dd5-4b00-a2f4-53e11abb1e9c None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:19.747596 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:19.797953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 237/459] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:38:19.799343 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858a663-e24d-40b3-953e-21fb406385ab None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:19.799818 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858a663-e24d-40b3-953e-21fb406385ab None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:19.800096 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858a663-e24d-40b3-953e-21fb406385ab None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:19.802283 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 238/460] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:19.809017 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09370be5-25d5-4621-b155-71ae3c0c46e0 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:19.809381 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09370be5-25d5-4621-b155-71ae3c0c46e0 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:19.809727 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09370be5-25d5-4621-b155-71ae3c0c46e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:19.829414 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-09370be5-25d5-4621-b155-71ae3c0c46e0 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:19.830743 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-09370be5-25d5-4621-b155-71ae3c0c46e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:19.837749 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab59bda-6dd5-4b00-a2f4-53e11abb1e9c None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:19.876509 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d08374a-bf0f-4e6e-add8-17c78b487dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:19.877238 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d08374a-bf0f-4e6e-add8-17c78b487dbb None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:19.877519 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d08374a-bf0f-4e6e-add8-17c78b487dbb None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:19.877794 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d08374a-bf0f-4e6e-add8-17c78b487dbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:19.884113 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:19.889166 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d08374a-bf0f-4e6e-add8-17c78b487dbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:19.890769 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d08374a-bf0f-4e6e-add8-17c78b487dbb None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:19.908008 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-2d08374a-bf0f-4e6e-add8-17c78b487dbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:19.908008 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:19.908008 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:19.908008 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:19.908008 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:19.908008 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:19.908008 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:19.908008 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:19.908008 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:19.908008 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:19.908008 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:19.908008 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:19.908008 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:19.908008 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:19.908008 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:19.908922 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:19.908922 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:19.908922 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:19.908922 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:19.908922 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:19.908922 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:19.908922 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:19.908922 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:19.908922 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:19.908922 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:19.908922 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:19.908922 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:19.908922 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:19.908922 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:19.908922 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:19.908922 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:19.908922 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:19.909744 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:19.909744 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:19.909744 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:19.909744 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:19.909744 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 223/461] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:19.910450 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d08374a-bf0f-4e6e-add8-17c78b487dbb None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:19.914015 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 239/462] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:38:19.916482 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09370be5-25d5-4621-b155-71ae3c0c46e0 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:20.017329 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:20.034007 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09370be5-25d5-4621-b155-71ae3c0c46e0 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:20.047409 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d08374a-bf0f-4e6e-add8-17c78b487dbb None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:20.072356 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76547f38-ec2e-413d-996a-f84d4dbc0971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:20.072861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76547f38-ec2e-413d-996a-f84d4dbc0971 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:20.073151 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76547f38-ec2e-413d-996a-f84d4dbc0971 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:20.073430 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76547f38-ec2e-413d-996a-f84d4dbc0971 None admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:20.074129 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76547f38-ec2e-413d-996a-f84d4dbc0971 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:20.074437 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76547f38-ec2e-413d-996a-f84d4dbc0971 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:20.075854 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76547f38-ec2e-413d-996a-f84d4dbc0971 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:20.082077 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541ac5d6-9c36-4e9e-b2be-16eb025a954a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:20.082934 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ac5d6-9c36-4e9e-b2be-16eb025a954a None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:20.083052 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ac5d6-9c36-4e9e-b2be-16eb025a954a None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:20.083276 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ac5d6-9c36-4e9e-b2be-16eb025a954a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:20.092997 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 240/463] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:20.092997 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76547f38-ec2e-413d-996a-f84d4dbc0971 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:20.098449 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541ac5d6-9c36-4e9e-b2be-16eb025a954a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:20.099939 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541ac5d6-9c36-4e9e-b2be-16eb025a954a None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:20.114865 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-541ac5d6-9c36-4e9e-b2be-16eb025a954a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:20.114865 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:20.114865 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:20.114865 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:20.114865 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:20.114865 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:20.114865 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:20.114865 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:20.114865 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:20.114865 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:20.114865 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:20.114865 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:20.114865 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:20.114865 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:20.114865 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:20.116281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:20.116281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:20.116281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:20.116281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:20.116281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:20.116281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:20.116281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:20.116281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:20.116281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:20.116281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:20.116281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:20.116281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:20.116281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:20.116281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:20.116281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:20.116281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:20.116281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:20.117206 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:20.117206 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:20.117206 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:20.117206 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:20.121508 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 224/464] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:20.122613 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b74b7-2f33-4a2e-8c56-86321c6bffba None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:20.122905 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b74b7-2f33-4a2e-8c56-86321c6bffba None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:20.123184 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b74b7-2f33-4a2e-8c56-86321c6bffba None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:20.127673 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 225/465] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:20.127673 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b74b7-2f33-4a2e-8c56-86321c6bffba None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:20.145288 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:20.220016 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76547f38-ec2e-413d-996a-f84d4dbc0971 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:20.240584 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b74b7-2f33-4a2e-8c56-86321c6bffba None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:20.255069 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87277f67-51c8-40f7-9857-c7241bdb24f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:20.255733 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87277f67-51c8-40f7-9857-c7241bdb24f5 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:20.256110 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87277f67-51c8-40f7-9857-c7241bdb24f5 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:20.256932 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87277f67-51c8-40f7-9857-c7241bdb24f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:20.267767 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87277f67-51c8-40f7-9857-c7241bdb24f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:20.268503 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:20.269315 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87277f67-51c8-40f7-9857-c7241bdb24f5 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:20.278549 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20df5d32-d779-41e7-b697-5272977f09a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:20.279173 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df5d32-d779-41e7-b697-5272977f09a2 None admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:20.279456 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df5d32-d779-41e7-b697-5272977f09a2 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:20.279761 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df5d32-d779-41e7-b697-5272977f09a2 None admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:20.280446 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20df5d32-d779-41e7-b697-5272977f09a2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:20.281886 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20df5d32-d779-41e7-b697-5272977f09a2 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:20.284773 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-87277f67-51c8-40f7-9857-c7241bdb24f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:20.284773 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:20.284773 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:20.284773 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:20.284773 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:20.284773 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:20.284773 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:20.284773 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:20.284773 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:20.284773 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:20.284773 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:20.284773 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:20.284773 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:20.284773 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:20.284773 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:20.285636 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:20.285636 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:20.285636 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:20.285636 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:20.285636 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:20.285636 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:20.285636 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:20.285636 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:20.285636 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:20.285636 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:20.285636 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:20.285636 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:20.285636 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:20.285636 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:20.285636 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:20.285636 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:20.285636 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:20.286476 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:20.286476 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:20.286476 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:20.286476 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:20.286476 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 241/466] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:20.287184 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87277f67-51c8-40f7-9857-c7241bdb24f5 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:20.314089 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 226/467] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Jul 26 12:38:20 2022] POST /identity/v3/users => generated 246 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:38:20.314089 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0034de-74c0-47b0-84af-bee0138520f7 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:20.314089 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0034de-74c0-47b0-84af-bee0138520f7 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:20.314089 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0034de-74c0-47b0-84af-bee0138520f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:20.327947 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-ec0034de-74c0-47b0-84af-bee0138520f7 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:20.329108 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-ec0034de-74c0-47b0-84af-bee0138520f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:20.421048 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:20.423785 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 227/468] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:38:20.424715 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0034de-74c0-47b0-84af-bee0138520f7 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:20.446781 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87277f67-51c8-40f7-9857-c7241bdb24f5 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:20.493939 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf939d5-5118-4c05-a3eb-24366f741ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:20.494575 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf939d5-5118-4c05-a3eb-24366f741ee6 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:20.494860 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf939d5-5118-4c05-a3eb-24366f741ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:20.495323 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf939d5-5118-4c05-a3eb-24366f741ee6 None admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:20.496852 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf939d5-5118-4c05-a3eb-24366f741ee6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:20.497346 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf939d5-5118-4c05-a3eb-24366f741ee6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:20.498767 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf939d5-5118-4c05-a3eb-24366f741ee6 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:20.515459 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 242/469] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:20.516970 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf939d5-5118-4c05-a3eb-24366f741ee6 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:20.562775 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0034de-74c0-47b0-84af-bee0138520f7 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:20.578417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:20.613660 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72fa90f-4b26-4bd4-8937-b41976b9006f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:20.614557 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72fa90f-4b26-4bd4-8937-b41976b9006f None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:20.614840 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72fa90f-4b26-4bd4-8937-b41976b9006f None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:20.615116 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72fa90f-4b26-4bd4-8937-b41976b9006f None admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:20.616364 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72fa90f-4b26-4bd4-8937-b41976b9006f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:20.616673 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72fa90f-4b26-4bd4-8937-b41976b9006f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:20.618071 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72fa90f-4b26-4bd4-8937-b41976b9006f None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:20.630180 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 228/470] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:20.631005 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72fa90f-4b26-4bd4-8937-b41976b9006f None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:20.649695 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf939d5-5118-4c05-a3eb-24366f741ee6 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:20.691468 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720de2a4-5c3d-4749-bade-58cd5249680d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:20.692514 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720de2a4-5c3d-4749-bade-58cd5249680d None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:20.692999 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720de2a4-5c3d-4749-bade-58cd5249680d None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:20.693500 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720de2a4-5c3d-4749-bade-58cd5249680d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:20.704681 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720de2a4-5c3d-4749-bade-58cd5249680d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:20.706477 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720de2a4-5c3d-4749-bade-58cd5249680d None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:20.722590 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-720de2a4-5c3d-4749-bade-58cd5249680d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:20.722590 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:20.722590 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:20.722590 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:20.722590 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:20.722590 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:20.722590 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:20.722590 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:20.722590 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:20.722590 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:20.722590 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:20.722590 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:20.722590 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:20.722590 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:20.722590 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:20.723477 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:20.723477 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:20.723477 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:20.723477 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:20.723477 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:20.723477 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:20.723477 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:20.723477 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:20.723477 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:20.723477 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:20.723477 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:20.723477 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:20.723477 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:20.723477 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:20.723477 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:20.723477 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:20.723477 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:20.724365 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:20.724365 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:20.724365 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:20.724365 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:20.726374 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 243/471] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:20.727393 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720de2a4-5c3d-4749-bade-58cd5249680d None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:20.730424 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:20.753585 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72fa90f-4b26-4bd4-8937-b41976b9006f None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:20.791711 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754df206-bc56-4aad-b6a2-f4263e85791a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:20.792227 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754df206-bc56-4aad-b6a2-f4263e85791a None admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:20.792506 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754df206-bc56-4aad-b6a2-f4263e85791a None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:20.792780 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754df206-bc56-4aad-b6a2-f4263e85791a None admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:20.793470 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754df206-bc56-4aad-b6a2-f4263e85791a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:20.794888 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754df206-bc56-4aad-b6a2-f4263e85791a None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:20.819444 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 229/472] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Jul 26 12:38:20 2022] POST /identity/v3/users => generated 248 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:38:20.820447 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754df206-bc56-4aad-b6a2-f4263e85791a None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:20.842812 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720de2a4-5c3d-4749-bade-58cd5249680d None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:20.855555 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:20.883685 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77bc34d-aeec-4ba4-9f83-378273ee9e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:20.884879 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77bc34d-aeec-4ba4-9f83-378273ee9e2c None admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:20.885374 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77bc34d-aeec-4ba4-9f83-378273ee9e2c None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:20.885883 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77bc34d-aeec-4ba4-9f83-378273ee9e2c None admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:20.886788 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77bc34d-aeec-4ba4-9f83-378273ee9e2c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:20.888547 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77bc34d-aeec-4ba4-9f83-378273ee9e2c None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:20.924828 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 244/473] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Jul 26 12:38:20 2022] POST /identity/v3/users => generated 248 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:38:20.926556 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520767e-d8db-4f24-9ebc-0954345cf77f None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:20.927058 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520767e-d8db-4f24-9ebc-0954345cf77f None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:20.927592 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520767e-d8db-4f24-9ebc-0954345cf77f None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:20.936149 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 245/474] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:20 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:20.941600 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b1012-9847-4880-a9c8-8d7a20632946 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:20.942283 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b1012-9847-4880-a9c8-8d7a20632946 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:20.942766 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b1012-9847-4880-a9c8-8d7a20632946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:20.962459 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754df206-bc56-4aad-b6a2-f4263e85791a None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:20.984871 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-f86b1012-9847-4880-a9c8-8d7a20632946 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:20.986140 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-f86b1012-9847-4880-a9c8-8d7a20632946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:21.009968 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d8889-aeb1-4494-9cb7-b68c638e699a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:21.010618 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d8889-aeb1-4494-9cb7-b68c638e699a None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:21.010961 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d8889-aeb1-4494-9cb7-b68c638e699a None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:21.011238 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d8889-aeb1-4494-9cb7-b68c638e699a None admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:21.012111 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d8889-aeb1-4494-9cb7-b68c638e699a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:21.012418 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d8889-aeb1-4494-9cb7-b68c638e699a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:21.013858 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d8889-aeb1-4494-9cb7-b68c638e699a None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:21.031726 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 230/475] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:21.044741 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d8889-aeb1-4494-9cb7-b68c638e699a None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:21.114646 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:21.141718 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 246/476] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 12:38:21.163538 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:21.170012 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86b1012-9847-4880-a9c8-8d7a20632946 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:21.175830 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d8889-aeb1-4494-9cb7-b68c638e699a None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:21.221218 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64312bc5-46a1-4190-8604-e1e17a9aec6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:21.222201 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64312bc5-46a1-4190-8604-e1e17a9aec6e None admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:21.222484 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64312bc5-46a1-4190-8604-e1e17a9aec6e None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:21.222761 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64312bc5-46a1-4190-8604-e1e17a9aec6e None admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:21.223446 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64312bc5-46a1-4190-8604-e1e17a9aec6e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:21.224923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64312bc5-46a1-4190-8604-e1e17a9aec6e None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:21.251822 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 231/477] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Jul 26 12:38:21 2022] POST /identity/v3/users => generated 249 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:38:21.298920 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86b1012-9847-4880-a9c8-8d7a20632946 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:21.346371 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b219e3-a2e4-49ca-94cd-3607e539cd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:21.347011 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b219e3-a2e4-49ca-94cd-3607e539cd66 None admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:21.347291 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b219e3-a2e4-49ca-94cd-3607e539cd66 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:21.347563 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b219e3-a2e4-49ca-94cd-3607e539cd66 None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:21.348319 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b219e3-a2e4-49ca-94cd-3607e539cd66 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:21.349701 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b219e3-a2e4-49ca-94cd-3607e539cd66 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:21.362226 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.sql.core [None req-f4b219e3-a2e4-49ca-94cd-3607e539cd66 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 26 12:38:21.362226 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 26 12:38:21.362226 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [parameters: {'id': 'd92613a0a1394d44b45bb16b3b1709f4', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 26 12:38:21.362226 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91748) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 26 12:38:21.389255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-f4b219e3-a2e4-49ca-94cd-3607e539cd66 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 26 12:38:21.389255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:21.389255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 12:38:21.389255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 12:38:21.389255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 12:38:21.389255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 12:38:21.389255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 12:38:21.389255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 12:38:21.389255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 12:38:21.389255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application conn.query(q) Jul 26 12:38:21.389255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 12:38:21.389255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 12:38:21.389255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 12:38:21.389255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application result.read() Jul 26 12:38:21.389255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 12:38:21.390231 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 12:38:21.390231 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 12:38:21.390231 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 12:38:21.390231 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 12:38:21.390231 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 12:38:21.390231 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 12:38:21.390231 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 12:38:21.390231 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 26 12:38:21.390231 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:21.390231 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 26 12:38:21.390231 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:21.390231 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:21.390231 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 26 12:38:21.390231 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 26 12:38:21.390231 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 26 12:38:21.390231 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return ref.to_dict() Jul 26 12:38:21.390231 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 12:38:21.390231 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application next(self.gen) Jul 26 12:38:21.391186 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 26 12:38:21.391186 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application yield resource Jul 26 12:38:21.391186 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 12:38:21.391186 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application next(self.gen) Jul 26 12:38:21.391186 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 26 12:38:21.391186 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application self.session.rollback() Jul 26 12:38:21.391186 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 26 12:38:21.391186 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application self.force_reraise() Jul 26 12:38:21.391186 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 26 12:38:21.391186 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise self.value Jul 26 12:38:21.391186 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 26 12:38:21.391186 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 26 12:38:21.391186 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 26 12:38:21.391186 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application session.commit() Jul 26 12:38:21.391186 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 26 12:38:21.391186 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 26 12:38:21.391186 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 26 12:38:21.392066 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application self._prepare_impl() Jul 26 12:38:21.392066 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 26 12:38:21.392066 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application self.session.flush() Jul 26 12:38:21.392066 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 26 12:38:21.392066 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application self._flush(objects) Jul 26 12:38:21.392066 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 26 12:38:21.392066 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 26 12:38:21.392066 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 26 12:38:21.392066 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application compat.raise_( Jul 26 12:38:21.392066 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 12:38:21.392066 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception Jul 26 12:38:21.392066 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 26 12:38:21.392066 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application flush_context.execute() Jul 26 12:38:21.392066 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 26 12:38:21.392066 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rec.execute(self) Jul 26 12:38:21.392066 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 26 12:38:21.392066 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 26 12:38:21.392933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 26 12:38:21.392933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 26 12:38:21.392933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 26 12:38:21.392933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 26 12:38:21.392933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 26 12:38:21.392933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 26 12:38:21.392933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 26 12:38:21.392933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 26 12:38:21.392933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 26 12:38:21.392933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 26 12:38:21.392933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 26 12:38:21.392933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 26 12:38:21.392933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 26 12:38:21.392933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 26 12:38:21.392933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 12:38:21.392933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception Jul 26 12:38:21.394760 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 12:38:21.394760 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 12:38:21.394760 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 12:38:21.394760 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 12:38:21.394760 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 12:38:21.394760 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 12:38:21.394760 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 12:38:21.394760 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application conn.query(q) Jul 26 12:38:21.394760 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 12:38:21.394760 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 12:38:21.394760 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 12:38:21.394760 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application result.read() Jul 26 12:38:21.394760 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 12:38:21.394760 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 12:38:21.394760 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 12:38:21.394760 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 12:38:21.398562 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 12:38:21.398562 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 12:38:21.398562 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 12:38:21.398562 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 12:38:21.398562 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 26 12:38:21.398562 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 26 12:38:21.398562 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [parameters: {'id': 'd92613a0a1394d44b45bb16b3b1709f4', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 26 12:38:21.398562 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 26 12:38:21.398562 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:21.398562 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 12:38:21.398562 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:21.398562 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:21.398562 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:21.398562 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:21.398562 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:21.398562 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:21.400559 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:21.400559 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:21.400559 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:21.400559 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:21.400559 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:21.400559 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:21.400559 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 26 12:38:21.400559 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 26 12:38:21.400559 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:21.400559 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:21.400559 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 26 12:38:21.400559 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 26 12:38:21.400559 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 26 12:38:21.400559 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 26 12:38:21.400559 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 26 12:38:21.400559 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:21.402590 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 247/478] 10.222.0.20 () {60 vars in 1131 bytes} [Tue Jul 26 12:38:21 2022] POST /identity/v3/roles => generated 144 bytes in 221 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:21.402766 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64312bc5-46a1-4190-8604-e1e17a9aec6e None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:21.537937 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64312bc5-46a1-4190-8604-e1e17a9aec6e None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:21.577223 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a5862f-ad1c-45ea-95f4-37fd456f34ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:21.578013 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a5862f-ad1c-45ea-95f4-37fd456f34ba None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:21.578199 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a5862f-ad1c-45ea-95f4-37fd456f34ba None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:21.578397 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a5862f-ad1c-45ea-95f4-37fd456f34ba None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:21.591180 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a5862f-ad1c-45ea-95f4-37fd456f34ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:38:21.596698 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-25a5862f-ad1c-45ea-95f4-37fd456f34ba None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 12:38:21.596698 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:21.596698 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:21.596698 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:21.596698 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:21.596698 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:21.596698 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:21.596698 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:21.596698 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:21.596698 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:21.596698 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:21.596698 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:21.596698 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:38:21.596698 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:38:21.596698 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:38:21.596698 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:38:21.598719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:38:21.598719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:38:21.598719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:38:21.598719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:38:21.598719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:21.598719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:21.598719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:21.598719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:21.598719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:21.598719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:21.598719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:21.598719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:21.598719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:21.598719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:21.598719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:21.598719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:21.599864 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:21.599864 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:21.599864 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:21.599864 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:21.599864 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:38:21.599864 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:38:21.599864 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:38:21.599864 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:38:21.599864 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:38:21.599864 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:38:21.599864 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 12:38:21.599864 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:21.599864 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 232/479] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 26 12:38:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:21.604382 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b219e3-a2e4-49ca-94cd-3607e539cd66 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:21.663206 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:21.720538 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b219e3-a2e4-49ca-94cd-3607e539cd66 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:21.761419 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38337409-9cfd-4e97-b130-dbff697d517c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:21.762091 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38337409-9cfd-4e97-b130-dbff697d517c None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:21.762370 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38337409-9cfd-4e97-b130-dbff697d517c None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:21.762696 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38337409-9cfd-4e97-b130-dbff697d517c None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:21.763424 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38337409-9cfd-4e97-b130-dbff697d517c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:21.763762 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38337409-9cfd-4e97-b130-dbff697d517c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:21.765140 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38337409-9cfd-4e97-b130-dbff697d517c None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:21.776191 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 248/480] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:38:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:22.821011 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152b292-7e06-4649-bc72-f849a115c180 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:22.821526 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152b292-7e06-4649-bc72-f849a115c180 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:22.821526 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152b292-7e06-4649-bc72-f849a115c180 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:22.823150 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 233/481] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:22.828676 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bfe103-9d0b-45ed-9026-1585c9b5c6b7 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:22.829117 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bfe103-9d0b-45ed-9026-1585c9b5c6b7 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:22.829308 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bfe103-9d0b-45ed-9026-1585c9b5c6b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:22.854420 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-53bfe103-9d0b-45ed-9026-1585c9b5c6b7 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:22.855618 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-53bfe103-9d0b-45ed-9026-1585c9b5c6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:22.956242 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 249/482] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 12:38:23.001082 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c152b292-7e06-4649-bc72-f849a115c180 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:23.009821 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:23.145467 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c152b292-7e06-4649-bc72-f849a115c180 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:23.193013 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a574c962-7f3c-4c9f-8aeb-a7583e3743eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:23.193704 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574c962-7f3c-4c9f-8aeb-a7583e3743eb None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:23.194056 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574c962-7f3c-4c9f-8aeb-a7583e3743eb None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:23.194330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574c962-7f3c-4c9f-8aeb-a7583e3743eb None admin] PATH_INFO: `/v3/roles/service` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:23.205400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a574c962-7f3c-4c9f-8aeb-a7583e3743eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:38:23.210415 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-a574c962-7f3c-4c9f-8aeb-a7583e3743eb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 12:38:23.210415 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:23.210415 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:23.210415 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:23.210415 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:23.210415 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:23.210415 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:23.210415 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:23.210415 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:23.210415 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:23.210415 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:23.210415 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:23.210415 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:38:23.210415 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:38:23.210415 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:38:23.210415 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:38:23.211271 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:38:23.211271 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:38:23.211271 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:38:23.211271 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:38:23.211271 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:23.211271 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:23.211271 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:23.211271 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:23.211271 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:23.211271 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:23.211271 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:23.211271 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:23.211271 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:23.211271 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:23.211271 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:23.211271 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:23.212165 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:23.212165 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:23.212165 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:23.212165 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:23.212165 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:38:23.212165 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:38:23.212165 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:38:23.212165 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:38:23.212165 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:38:23.212165 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:38:23.212165 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 12:38:23.212165 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:23.215744 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 234/483] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 26 12:38:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:23.217792 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bfe103-9d0b-45ed-9026-1585c9b5c6b7 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:23.264566 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:23.341043 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bfe103-9d0b-45ed-9026-1585c9b5c6b7 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:23.344149 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71a74a-272e-49d9-a13e-f966b8dd49b4 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:23.344437 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71a74a-272e-49d9-a13e-f966b8dd49b4 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:23.344717 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71a74a-272e-49d9-a13e-f966b8dd49b4 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:23.351757 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 235/484] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:23.357068 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f91ac1-45f8-4d43-a93f-807f0a0400f0 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:23.357350 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f91ac1-45f8-4d43-a93f-807f0a0400f0 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:23.357625 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f91ac1-45f8-4d43-a93f-807f0a0400f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:23.385860 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-92f91ac1-45f8-4d43-a93f-807f0a0400f0 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:23.387039 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-92f91ac1-45f8-4d43-a93f-807f0a0400f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:23.390212 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9324e3a-122b-42c3-81c2-4905c9c11e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:23.390833 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9324e3a-122b-42c3-81c2-4905c9c11e81 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:23.391137 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9324e3a-122b-42c3-81c2-4905c9c11e81 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:23.391380 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9324e3a-122b-42c3-81c2-4905c9c11e81 None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:23.392120 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9324e3a-122b-42c3-81c2-4905c9c11e81 None admin] RBAC: Adding query filter params (name=service) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:23.392424 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9324e3a-122b-42c3-81c2-4905c9c11e81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:23.393785 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9324e3a-122b-42c3-81c2-4905c9c11e81 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:23.409760 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 250/485] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Jul 26 12:38:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:23.415664 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9324e3a-122b-42c3-81c2-4905c9c11e81 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:23.526085 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:23.540150 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 236/486] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:38:23.541848 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded93907-d5cd-4b29-aa9d-d226405cf6b8 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:23.542134 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded93907-d5cd-4b29-aa9d-d226405cf6b8 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:23.542459 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded93907-d5cd-4b29-aa9d-d226405cf6b8 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:23.544453 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 237/487] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:23.549826 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb4e7f8-8d22-48e4-8ee5-eae56925641c None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:23.550108 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb4e7f8-8d22-48e4-8ee5-eae56925641c None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:23.550378 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb4e7f8-8d22-48e4-8ee5-eae56925641c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:23.573957 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9324e3a-122b-42c3-81c2-4905c9c11e81 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:23.576667 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-2eb4e7f8-8d22-48e4-8ee5-eae56925641c None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:23.578022 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-2eb4e7f8-8d22-48e4-8ee5-eae56925641c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:23.582890 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:23.621144 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3bb742-9ceb-420d-94cc-6d49b854cbfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:23.622088 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3bb742-9ceb-420d-94cc-6d49b854cbfc None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:23.622547 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3bb742-9ceb-420d-94cc-6d49b854cbfc None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:23.622995 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3bb742-9ceb-420d-94cc-6d49b854cbfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:23.637846 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3bb742-9ceb-420d-94cc-6d49b854cbfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:23.640480 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3bb742-9ceb-420d-94cc-6d49b854cbfc None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:23.658621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-1a3bb742-9ceb-420d-94cc-6d49b854cbfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:23.658621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:23.658621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:23.658621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:23.658621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:23.658621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:23.658621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:23.658621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:23.658621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:23.658621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:23.658621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:23.658621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:23.658621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:23.658621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:23.658621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:23.659486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:23.659486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:23.659486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:23.659486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:23.659486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:23.659486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:23.659486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:23.659486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:23.659486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:23.659486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:23.659486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:23.659486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:23.659486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:23.659486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:23.659486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:23.659486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:23.659486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:23.660402 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:23.660402 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:23.660402 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:23.660402 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:23.660631 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 238/488] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:38:23.660631 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb4e7f8-8d22-48e4-8ee5-eae56925641c None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:23.667294 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 251/489] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:38:23.669801 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3bb742-9ceb-420d-94cc-6d49b854cbfc None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:23.795443 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb4e7f8-8d22-48e4-8ee5-eae56925641c None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:23.804294 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3bb742-9ceb-420d-94cc-6d49b854cbfc None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:23.830697 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:23.842872 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a69cdc-012c-440e-bbfc-b09d2941296c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:23.843653 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a69cdc-012c-440e-bbfc-b09d2941296c None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:23.844042 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a69cdc-012c-440e-bbfc-b09d2941296c None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:23.844422 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a69cdc-012c-440e-bbfc-b09d2941296c None admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:23.845289 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a69cdc-012c-440e-bbfc-b09d2941296c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:23.845682 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a69cdc-012c-440e-bbfc-b09d2941296c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:23.847289 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a69cdc-012c-440e-bbfc-b09d2941296c None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:23.847692 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53eb1c-6082-4c2b-a3d1-62191dfc9109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:23.848492 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53eb1c-6082-4c2b-a3d1-62191dfc9109 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:23.849216 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53eb1c-6082-4c2b-a3d1-62191dfc9109 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:23.849640 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53eb1c-6082-4c2b-a3d1-62191dfc9109 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:23.869091 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 252/490] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:38:23.869091 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a69cdc-012c-440e-bbfc-b09d2941296c None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:23.870930 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f53eb1c-6082-4c2b-a3d1-62191dfc9109 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:38:23.877703 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-1f53eb1c-6082-4c2b-a3d1-62191dfc9109 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 12:38:23.877703 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:23.877703 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:23.877703 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:23.877703 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:23.877703 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:23.877703 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:23.877703 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:23.877703 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:23.877703 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:23.877703 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:23.877703 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:23.877703 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:38:23.877703 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:38:23.877703 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:38:23.877703 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:38:23.878624 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:38:23.878624 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:38:23.878624 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:38:23.878624 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:38:23.878624 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:23.878624 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:23.878624 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:23.878624 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:23.878624 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:23.878624 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:23.878624 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:23.878624 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:23.878624 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:23.878624 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:23.878624 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:23.878624 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:23.880209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:23.880209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:23.880209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:23.880209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:23.880209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:38:23.880209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:38:23.880209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:38:23.880209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:38:23.880209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:38:23.880209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:38:23.880209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 12:38:23.880209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:23.883705 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 239/491] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 26 12:38:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:23.884783 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53eb1c-6082-4c2b-a3d1-62191dfc9109 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:23.902155 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:24.004073 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a69cdc-012c-440e-bbfc-b09d2941296c None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:24.010987 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53eb1c-6082-4c2b-a3d1-62191dfc9109 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:24.040852 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf45bd-3f04-4d18-aa8c-07d9af5b270c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:24.041540 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf45bd-3f04-4d18-aa8c-07d9af5b270c None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:24.041943 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf45bd-3f04-4d18-aa8c-07d9af5b270c None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:24.042185 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf45bd-3f04-4d18-aa8c-07d9af5b270c None admin] PATH_INFO: `/v3/roles/service` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:24.051283 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a53d9b-2a99-4958-8017-45ba6a7e35e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:24.052276 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a53d9b-2a99-4958-8017-45ba6a7e35e9 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:24.052779 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a53d9b-2a99-4958-8017-45ba6a7e35e9 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:24.053823 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a53d9b-2a99-4958-8017-45ba6a7e35e9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:24.054387 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baf45bd-3f04-4d18-aa8c-07d9af5b270c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:38:24.059798 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-4baf45bd-3f04-4d18-aa8c-07d9af5b270c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 12:38:24.059798 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:24.059798 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:24.059798 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:24.059798 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:24.059798 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:24.059798 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:24.059798 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:24.059798 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:24.059798 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:24.059798 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:24.059798 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:24.059798 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:38:24.059798 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:38:24.059798 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:38:24.059798 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:38:24.060674 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:38:24.060674 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:38:24.060674 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:38:24.060674 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:38:24.060674 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:24.060674 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:24.060674 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:24.060674 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:24.060674 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:24.060674 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:24.060674 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:24.060674 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:24.060674 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:24.060674 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:24.060674 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:24.060674 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:24.061469 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:24.061469 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:24.061469 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:24.061469 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:24.061469 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:38:24.061469 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:38:24.061469 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:38:24.061469 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:38:24.061469 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:38:24.061469 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:38:24.061469 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 12:38:24.061469 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:24.065157 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a53d9b-2a99-4958-8017-45ba6a7e35e9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:24.065910 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 253/492] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 26 12:38:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:24.066763 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf45bd-3f04-4d18-aa8c-07d9af5b270c None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:24.067570 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a53d9b-2a99-4958-8017-45ba6a7e35e9 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:24.084209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-91a53d9b-2a99-4958-8017-45ba6a7e35e9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 12:38:24.084209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:24.084209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:24.084209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:24.084209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:24.084209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:24.084209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:24.084209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:24.084209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:24.084209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:24.084209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:24.084209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:24.084209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 12:38:24.084209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 12:38:24.084209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 12:38:24.084209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 12:38:24.085314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:24.085314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:24.085314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 12:38:24.085314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:24.085314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 12:38:24.085314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:24.085314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:24.085314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:24.085314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:24.085314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:24.085314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:24.085314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:24.085314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:24.085314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:24.085314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:24.085314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:24.085314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:24.085314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:24.086217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:24.086217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:24.086217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 12:38:24.086217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 12:38:24.086217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 12:38:24.086217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 12:38:24.086217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 12:38:24.086217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 12:38:24.086217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 12:38:24.086217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:24.088732 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 240/493] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Jul 26 12:38:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:24.093393 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a53d9b-2a99-4958-8017-45ba6a7e35e9 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:24.145638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:24.151189 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:24.175523 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf45bd-3f04-4d18-aa8c-07d9af5b270c None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:24.211960 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a53d9b-2a99-4958-8017-45ba6a7e35e9 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:24.215441 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3106da93-f4a2-4489-9a7a-658ba9d9a26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:24.216112 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106da93-f4a2-4489-9a7a-658ba9d9a26b None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:24.216395 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106da93-f4a2-4489-9a7a-658ba9d9a26b None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:24.216665 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106da93-f4a2-4489-9a7a-658ba9d9a26b None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:24.217507 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3106da93-f4a2-4489-9a7a-658ba9d9a26b None admin] RBAC: Adding query filter params (name=service) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:24.217812 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3106da93-f4a2-4489-9a7a-658ba9d9a26b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:24.219248 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3106da93-f4a2-4489-9a7a-658ba9d9a26b None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:24.235698 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 254/494] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Jul 26 12:38:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:24.235698 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210adfd-c222-4eb9-8d54-4162eb000635 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:24.235698 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210adfd-c222-4eb9-8d54-4162eb000635 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:24.235698 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210adfd-c222-4eb9-8d54-4162eb000635 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:24.237861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 255/495] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:24.238195 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7210adfd-c222-4eb9-8d54-4162eb000635 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:24.260533 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d1071-6f76-48c0-9183-29a962dbbbc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:24.261547 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d1071-6f76-48c0-9183-29a962dbbbc8 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:24.262008 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d1071-6f76-48c0-9183-29a962dbbbc8 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:24.262286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d1071-6f76-48c0-9183-29a962dbbbc8 None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:24.263015 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09d1071-6f76-48c0-9183-29a962dbbbc8 None admin] RBAC: Adding query filter params (name=service) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:24.263322 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09d1071-6f76-48c0-9183-29a962dbbbc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:24.264738 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09d1071-6f76-48c0-9183-29a962dbbbc8 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:24.274443 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 241/496] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Jul 26 12:38:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:24.275758 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df50e655-aca3-48a9-92d7-5a659c5c91d9 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:24.276047 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df50e655-aca3-48a9-92d7-5a659c5c91d9 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:24.276322 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df50e655-aca3-48a9-92d7-5a659c5c91d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:24.296923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-df50e655-aca3-48a9-92d7-5a659c5c91d9 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:24.298106 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-df50e655-aca3-48a9-92d7-5a659c5c91d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:24.360226 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7210adfd-c222-4eb9-8d54-4162eb000635 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:24.366948 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 242/497] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:38:24.367823 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df50e655-aca3-48a9-92d7-5a659c5c91d9 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:24.404358 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:24.414002 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd79d85-3655-4c87-aecf-34cf38b6d164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:24.414708 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd79d85-3655-4c87-aecf-34cf38b6d164 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:24.414907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd79d85-3655-4c87-aecf-34cf38b6d164 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:24.415158 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd79d85-3655-4c87-aecf-34cf38b6d164 None admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:24.415987 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd79d85-3655-4c87-aecf-34cf38b6d164 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:24.416294 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd79d85-3655-4c87-aecf-34cf38b6d164 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:24.417744 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd79d85-3655-4c87-aecf-34cf38b6d164 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:24.444444 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:24.455676 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 256/498] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Jul 26 12:38:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:24.456386 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd79d85-3655-4c87-aecf-34cf38b6d164 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:24.479497 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df50e655-aca3-48a9-92d7-5a659c5c91d9 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:24.513102 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1882fa11-7e47-4719-b539-508a2e63bd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:24.513667 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1882fa11-7e47-4719-b539-508a2e63bd22 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:24.513943 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1882fa11-7e47-4719-b539-508a2e63bd22 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:24.514217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1882fa11-7e47-4719-b539-508a2e63bd22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:24.525015 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1882fa11-7e47-4719-b539-508a2e63bd22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:24.526576 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1882fa11-7e47-4719-b539-508a2e63bd22 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:24.551406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-1882fa11-7e47-4719-b539-508a2e63bd22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:24.551406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:24.551406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:24.551406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:24.551406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:24.551406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:24.551406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:24.551406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:24.551406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:24.551406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:24.551406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:24.551406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:24.551406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:24.551406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:24.551406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:24.552362 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:24.552362 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:24.552362 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:24.552362 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:24.552362 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:24.552362 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:24.552362 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:24.552362 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:24.552362 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:24.552362 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:24.552362 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:24.552362 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:24.552362 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:24.552362 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:24.552362 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:24.552362 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:24.552362 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:24.553227 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:24.553227 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:24.553227 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:24.553227 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:24.553227 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 243/499] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:24.553931 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1882fa11-7e47-4719-b539-508a2e63bd22 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:24.560235 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd79d85-3655-4c87-aecf-34cf38b6d164 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:24.620193 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256980d6-c212-49c3-b69e-0207c308476f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:24.620829 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256980d6-c212-49c3-b69e-0207c308476f None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:24.621111 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256980d6-c212-49c3-b69e-0207c308476f None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:24.621385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256980d6-c212-49c3-b69e-0207c308476f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:24.632026 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256980d6-c212-49c3-b69e-0207c308476f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:24.633518 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256980d6-c212-49c3-b69e-0207c308476f None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:24.647777 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:24.648874 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-256980d6-c212-49c3-b69e-0207c308476f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:24.648874 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:24.648874 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:24.648874 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:24.648874 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:24.648874 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:24.648874 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:24.648874 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:24.648874 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:24.648874 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:24.648874 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:24.648874 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:24.648874 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:24.648874 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:24.648874 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:24.650349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:24.650349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:24.650349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:24.650349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:24.650349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:24.650349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:24.650349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:24.650349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:24.650349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:24.650349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:24.650349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:24.650349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:24.650349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:24.650349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:24.650349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:24.650349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:24.650349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:24.651221 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:24.651221 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:24.651221 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:24.651221 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:24.651221 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 257/500] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:24.651221 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256980d6-c212-49c3-b69e-0207c308476f None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:24.665325 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1882fa11-7e47-4719-b539-508a2e63bd22 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:24.699503 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6eb140-e00b-4748-9329-2cb07a06a388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:24.700057 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:24.700171 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6eb140-e00b-4748-9329-2cb07a06a388 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:24.700448 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6eb140-e00b-4748-9329-2cb07a06a388 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:24.700999 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6eb140-e00b-4748-9329-2cb07a06a388 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:24.712429 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6eb140-e00b-4748-9329-2cb07a06a388 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:38:24.717733 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-df6eb140-e00b-4748-9329-2cb07a06a388 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 12:38:24.717733 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:24.717733 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:24.717733 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:24.717733 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:24.717733 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:24.717733 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:24.717733 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:24.717733 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:24.717733 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:24.717733 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:24.717733 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:24.717733 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:38:24.717733 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:38:24.717733 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:38:24.717733 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:38:24.718735 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:38:24.718735 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:38:24.718735 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:38:24.718735 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:38:24.718735 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:24.718735 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:24.718735 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:24.718735 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:24.718735 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:24.718735 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:24.718735 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:24.718735 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:24.718735 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:24.718735 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:24.718735 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:24.718735 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:24.719582 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:24.719582 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:24.719582 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:24.719582 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:24.719582 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:38:24.719582 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:38:24.719582 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:38:24.719582 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:38:24.719582 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:38:24.719582 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:38:24.719582 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 12:38:24.719582 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:24.719582 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 244/501] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 26 12:38:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:24.720312 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621f49e-3459-4104-bb50-9c4872711a16 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:24.720490 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621f49e-3459-4104-bb50-9c4872711a16 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:24.720751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621f49e-3459-4104-bb50-9c4872711a16 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:24.721943 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 245/502] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:24.722614 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7621f49e-3459-4104-bb50-9c4872711a16 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:24.755619 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256980d6-c212-49c3-b69e-0207c308476f None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:24.796366 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a939dc8-5a6f-4a50-be71-818a5a7c298a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:24.797082 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a939dc8-5a6f-4a50-be71-818a5a7c298a None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:24.797513 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a939dc8-5a6f-4a50-be71-818a5a7c298a None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:24.797936 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a939dc8-5a6f-4a50-be71-818a5a7c298a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:24.825440 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7621f49e-3459-4104-bb50-9c4872711a16 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:24.858703 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ac336-f13f-493c-83a4-42c6e3ab72fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:24.859317 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ac336-f13f-493c-83a4-42c6e3ab72fb None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:24.859577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ac336-f13f-493c-83a4-42c6e3ab72fb None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:24.859881 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ac336-f13f-493c-83a4-42c6e3ab72fb None admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:24.860569 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ac336-f13f-493c-83a4-42c6e3ab72fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:24.860876 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ac336-f13f-493c-83a4-42c6e3ab72fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:24.862262 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ac336-f13f-493c-83a4-42c6e3ab72fb None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:24.866729 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a939dc8-5a6f-4a50-be71-818a5a7c298a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:24.868300 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a939dc8-5a6f-4a50-be71-818a5a7c298a None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:24.875247 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 246/503] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:24.876105 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ac336-f13f-493c-83a4-42c6e3ab72fb None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:24.887000 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:24.926676 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:24.979484 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ac336-f13f-493c-83a4-42c6e3ab72fb None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:24.985173 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 258/504] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:38:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 12:38:24.985173 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a939dc8-5a6f-4a50-be71-818a5a7c298a None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:25.019597 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a8df90-1ea6-48ef-a7fd-a10d65e107b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:25.020244 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8df90-1ea6-48ef-a7fd-a10d65e107b0 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:25.020521 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8df90-1ea6-48ef-a7fd-a10d65e107b0 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:25.020791 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8df90-1ea6-48ef-a7fd-a10d65e107b0 None admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:25.021474 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a8df90-1ea6-48ef-a7fd-a10d65e107b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:25.021778 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a8df90-1ea6-48ef-a7fd-a10d65e107b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:25.023768 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a8df90-1ea6-48ef-a7fd-a10d65e107b0 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:25.035387 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 247/505] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:25.036677 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12584b40-e33e-4dc1-a7a4-7cc0d8cc23ad None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:25.036965 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12584b40-e33e-4dc1-a7a4-7cc0d8cc23ad None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:25.037237 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12584b40-e33e-4dc1-a7a4-7cc0d8cc23ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:25.056189 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-12584b40-e33e-4dc1-a7a4-7cc0d8cc23ad None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:25.057747 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-12584b40-e33e-4dc1-a7a4-7cc0d8cc23ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:25.081988 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a939dc8-5a6f-4a50-be71-818a5a7c298a None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:25.117383 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0f2b7-951c-453a-9a2e-291c68d25b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:25.118079 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0f2b7-951c-453a-9a2e-291c68d25b41 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:25.118366 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0f2b7-951c-453a-9a2e-291c68d25b41 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:25.118642 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0f2b7-951c-453a-9a2e-291c68d25b41 None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:25.119350 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a0f2b7-951c-453a-9a2e-291c68d25b41 None admin] RBAC: Adding query filter params (name=service) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:25.119721 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a0f2b7-951c-453a-9a2e-291c68d25b41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:25.121045 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a0f2b7-951c-453a-9a2e-291c68d25b41 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:25.121622 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:25.126905 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 248/506] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:38:25.127749 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12584b40-e33e-4dc1-a7a4-7cc0d8cc23ad None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:25.130096 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 259/507] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Jul 26 12:38:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:25.130896 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0f2b7-951c-453a-9a2e-291c68d25b41 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:25.149002 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:25.238529 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0f2b7-951c-453a-9a2e-291c68d25b41 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:25.244386 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12584b40-e33e-4dc1-a7a4-7cc0d8cc23ad None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:25.273156 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e0d23-1890-489b-82c1-b71331c505d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:25.273793 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e0d23-1890-489b-82c1-b71331c505d4 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:25.274076 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e0d23-1890-489b-82c1-b71331c505d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:25.274575 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e0d23-1890-489b-82c1-b71331c505d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:25.278966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1885462f-5115-475b-9556-877e2c2aff69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:25.279594 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1885462f-5115-475b-9556-877e2c2aff69 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:25.279900 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1885462f-5115-475b-9556-877e2c2aff69 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:25.280168 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1885462f-5115-475b-9556-877e2c2aff69 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:25.285071 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7e0d23-1890-489b-82c1-b71331c505d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:25.286607 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7e0d23-1890-489b-82c1-b71331c505d4 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:25.287213 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1885462f-5115-475b-9556-877e2c2aff69 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:25.288737 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1885462f-5115-475b-9556-877e2c2aff69 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:25.300270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-1885462f-5115-475b-9556-877e2c2aff69 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 12:38:25.300270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:25.300270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:25.300270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:25.300270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:25.300270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:25.300270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:25.300270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:25.300270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:25.300270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:25.300270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:25.300270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:25.300270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 12:38:25.300270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 12:38:25.300270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 12:38:25.300270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 12:38:25.301155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:25.301155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:25.301155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 12:38:25.301155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:25.301155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 12:38:25.301155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:25.301155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:25.301155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:25.301155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:25.301155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:25.301155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:25.301155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:25.301155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:25.301155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:25.301155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:25.301155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:25.301155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:25.301155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:25.302326 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:25.302326 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:25.302326 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 12:38:25.302326 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 12:38:25.302326 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 12:38:25.302326 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 12:38:25.302326 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 12:38:25.302326 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 12:38:25.302326 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 12:38:25.302326 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:25.302975 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-1e7e0d23-1890-489b-82c1-b71331c505d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:25.302975 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:25.302975 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:25.302975 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:25.302975 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:25.302975 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:25.302975 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:25.302975 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:25.302975 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:25.302975 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:25.302975 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:25.302975 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:25.302975 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:25.302975 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:25.302975 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:25.304597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:25.304597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:25.304597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:25.304597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:25.304597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:25.304597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:25.304597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:25.304597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:25.304597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:25.304597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:25.304597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:25.304597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:25.304597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:25.304597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:25.304597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:25.304597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:25.304597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:25.305463 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:25.305463 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:25.305463 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:25.305463 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:25.305669 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 249/508] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:38:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:25.305744 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 260/509] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:25.305818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1885462f-5115-475b-9556-877e2c2aff69 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:25.306401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e0d23-1890-489b-82c1-b71331c505d4 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:25.362483 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:25.370922 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:25.406554 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1885462f-5115-475b-9556-877e2c2aff69 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:25.409371 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e0d23-1890-489b-82c1-b71331c505d4 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:25.439833 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7035c3e-1d56-4bb4-9627-f99d9b9741a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:25.440462 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7035c3e-1d56-4bb4-9627-f99d9b9741a9 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:25.440746 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7035c3e-1d56-4bb4-9627-f99d9b9741a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:25.441020 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7035c3e-1d56-4bb4-9627-f99d9b9741a9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:25.442787 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b0c1b2-5f16-4ded-bf71-cbd702669acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:25.443414 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0c1b2-5f16-4ded-bf71-cbd702669acf None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:25.444320 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0c1b2-5f16-4ded-bf71-cbd702669acf None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:25.444600 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0c1b2-5f16-4ded-bf71-cbd702669acf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:25.449214 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7035c3e-1d56-4bb4-9627-f99d9b9741a9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:25.450692 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7035c3e-1d56-4bb4-9627-f99d9b9741a9 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:25.454979 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b0c1b2-5f16-4ded-bf71-cbd702669acf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:25.456446 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b0c1b2-5f16-4ded-bf71-cbd702669acf None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:25.462000 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-a7035c3e-1d56-4bb4-9627-f99d9b9741a9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 26 12:38:25.462000 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:25.462000 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:25.462000 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:25.462000 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:25.462000 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:25.462000 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:25.462000 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:25.462000 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:25.462000 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:25.462000 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:25.462000 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:25.462000 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 12:38:25.462000 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 12:38:25.462000 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 12:38:25.462000 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 12:38:25.462894 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:25.462894 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:25.462894 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 12:38:25.462894 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:25.462894 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 12:38:25.462894 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:25.462894 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:25.462894 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:25.462894 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:25.462894 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:25.462894 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:25.462894 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:25.462894 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:25.462894 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:25.462894 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:25.462894 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:25.462894 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:25.462894 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:25.463989 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:25.463989 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:25.463989 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 12:38:25.463989 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 12:38:25.463989 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 12:38:25.463989 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 12:38:25.463989 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 12:38:25.463989 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 12:38:25.463989 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 26 12:38:25.463989 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:25.463989 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 250/510] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:38:25 2022] GET /identity/v3/users/glance => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:25.463989 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7035c3e-1d56-4bb4-9627-f99d9b9741a9 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:25.473502 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-55b0c1b2-5f16-4ded-bf71-cbd702669acf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:25.473502 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:25.473502 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:25.473502 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:25.473502 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:25.473502 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:25.473502 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:25.473502 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:25.473502 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:25.473502 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:25.473502 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:25.473502 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:25.473502 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:25.473502 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:25.473502 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:25.474378 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:25.474378 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:25.474378 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:25.474378 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:25.474378 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:25.474378 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:25.474378 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:25.474378 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:25.474378 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:25.474378 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:25.474378 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:25.474378 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:25.474378 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:25.474378 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:25.474378 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:25.474378 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:25.474378 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:25.475789 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:25.475789 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:25.475789 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:25.475789 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:25.475789 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 261/511] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:25.477076 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b0c1b2-5f16-4ded-bf71-cbd702669acf None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:25.563935 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7035c3e-1d56-4bb4-9627-f99d9b9741a9 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:25.579237 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b0c1b2-5f16-4ded-bf71-cbd702669acf None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:25.580525 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:25.595697 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:25.597639 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a342f64-a0c4-4e4c-9852-ae39a7f6a377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:25.598537 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a342f64-a0c4-4e4c-9852-ae39a7f6a377 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:25.598907 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a342f64-a0c4-4e4c-9852-ae39a7f6a377 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:25.599244 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a342f64-a0c4-4e4c-9852-ae39a7f6a377 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:25.610272 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a342f64-a0c4-4e4c-9852-ae39a7f6a377 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:25.611748 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a342f64-a0c4-4e4c-9852-ae39a7f6a377 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:25.615325 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dde7c4-da37-4e1f-b21a-7dad978d4374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:25.616546 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dde7c4-da37-4e1f-b21a-7dad978d4374 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:25.616831 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dde7c4-da37-4e1f-b21a-7dad978d4374 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:25.617106 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dde7c4-da37-4e1f-b21a-7dad978d4374 None admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:25.617830 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dde7c4-da37-4e1f-b21a-7dad978d4374 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:25.618138 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dde7c4-da37-4e1f-b21a-7dad978d4374 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:25.619516 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dde7c4-da37-4e1f-b21a-7dad978d4374 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:25.626819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-6a342f64-a0c4-4e4c-9852-ae39a7f6a377 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:25.626819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:25.626819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:25.626819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:25.626819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:25.626819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:25.626819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:25.626819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:25.626819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:25.626819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:25.626819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:25.626819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:25.626819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:25.626819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:25.626819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:25.627682 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:25.627682 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:25.627682 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:25.627682 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:25.627682 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:25.627682 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:25.627682 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:25.627682 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:25.627682 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:25.627682 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:25.627682 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:25.627682 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:25.627682 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:25.627682 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:25.627682 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:25.627682 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:25.627682 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:25.628531 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:25.628531 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:25.628531 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:25.628531 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:25.628531 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 251/512] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:25.628810 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a342f64-a0c4-4e4c-9852-ae39a7f6a377 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:25.632702 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 262/513] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:25.633856 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dde7c4-da37-4e1f-b21a-7dad978d4374 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:25.729494 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a342f64-a0c4-4e4c-9852-ae39a7f6a377 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:25.731778 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dde7c4-da37-4e1f-b21a-7dad978d4374 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:25.762682 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd70c355-d2bc-4140-99bb-e561f7e43d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:25.763294 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70c355-d2bc-4140-99bb-e561f7e43d28 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:25.763577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70c355-d2bc-4140-99bb-e561f7e43d28 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:25.763884 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70c355-d2bc-4140-99bb-e561f7e43d28 None admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:25.764678 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd70c355-d2bc-4140-99bb-e561f7e43d28 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:25.764989 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd70c355-d2bc-4140-99bb-e561f7e43d28 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:25.765570 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e647e7c-504d-4bee-9a64-d7966bae5154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:25.766174 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e647e7c-504d-4bee-9a64-d7966bae5154 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:25.766433 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd70c355-d2bc-4140-99bb-e561f7e43d28 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:25.766523 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e647e7c-504d-4bee-9a64-d7966bae5154 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:25.766705 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e647e7c-504d-4bee-9a64-d7966bae5154 None admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:25.767483 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e647e7c-504d-4bee-9a64-d7966bae5154 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:25.767825 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e647e7c-504d-4bee-9a64-d7966bae5154 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:25.769253 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e647e7c-504d-4bee-9a64-d7966bae5154 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:25.793116 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 263/514] 10.222.0.20 () {56 vars in 1137 bytes} [Tue Jul 26 12:38:25 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:25.793116 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e647e7c-504d-4bee-9a64-d7966bae5154 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:25.806854 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 252/515] 10.222.0.20 () {56 vars in 1137 bytes} [Tue Jul 26 12:38:25 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:25.807730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd70c355-d2bc-4140-99bb-e561f7e43d28 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:25.828171 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:25.835412 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:25.895352 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e647e7c-504d-4bee-9a64-d7966bae5154 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:25.909320 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd70c355-d2bc-4140-99bb-e561f7e43d28 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:25.935339 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea5c256-5387-431a-b37e-ad5e019d1f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:25.936012 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea5c256-5387-431a-b37e-ad5e019d1f68 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:25.936299 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea5c256-5387-431a-b37e-ad5e019d1f68 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:25.936571 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea5c256-5387-431a-b37e-ad5e019d1f68 None admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:25.937268 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea5c256-5387-431a-b37e-ad5e019d1f68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:25.937579 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea5c256-5387-431a-b37e-ad5e019d1f68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:25.938985 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea5c256-5387-431a-b37e-ad5e019d1f68 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:25.950276 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 264/516] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:25.951065 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea5c256-5387-431a-b37e-ad5e019d1f68 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:25.952467 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f7199e-6a51-402e-aa80-469a2fd79db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:25.953089 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7199e-6a51-402e-aa80-469a2fd79db0 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:25.953370 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7199e-6a51-402e-aa80-469a2fd79db0 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:25.953644 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7199e-6a51-402e-aa80-469a2fd79db0 None admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:25.954338 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f7199e-6a51-402e-aa80-469a2fd79db0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:25.954641 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f7199e-6a51-402e-aa80-469a2fd79db0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:25.956051 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f7199e-6a51-402e-aa80-469a2fd79db0 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:25.968128 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 253/517] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:25.969625 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f7199e-6a51-402e-aa80-469a2fd79db0 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:26.047287 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:26.055057 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea5c256-5387-431a-b37e-ad5e019d1f68 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:26.057771 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:26.069911 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f7199e-6a51-402e-aa80-469a2fd79db0 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:26.088973 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e71073-4b14-4645-b8f6-f8944b35182d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:26.089547 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e71073-4b14-4645-b8f6-f8944b35182d None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:26.089831 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e71073-4b14-4645-b8f6-f8944b35182d None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:26.090109 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e71073-4b14-4645-b8f6-f8944b35182d None admin] PATH_INFO: `/v3/projects/service` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:26.097206 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e71073-4b14-4645-b8f6-f8944b35182d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:26.098661 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e71073-4b14-4645-b8f6-f8944b35182d None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:26.103459 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef38d32-235d-482e-a0e0-0a22bf596677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:26.104116 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef38d32-235d-482e-a0e0-0a22bf596677 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:26.104404 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef38d32-235d-482e-a0e0-0a22bf596677 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:26.104679 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef38d32-235d-482e-a0e0-0a22bf596677 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:26.110612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-90e71073-4b14-4645-b8f6-f8944b35182d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:26.110612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:26.110612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:26.110612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:26.110612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:26.110612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:26.110612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:26.110612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:26.110612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:26.110612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:26.110612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:26.110612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:26.110612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 12:38:26.110612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 12:38:26.110612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 12:38:26.111595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 12:38:26.111595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:26.111595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:26.111595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:26.111595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:26.111595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:26.111595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:26.111595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:26.111595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:26.111595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:26.111595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:26.111595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:26.111595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:26.111595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:26.111595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:26.111595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:26.111595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:26.112485 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 12:38:26.112485 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 12:38:26.112485 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 12:38:26.112485 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 12:38:26.112485 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 12:38:26.112485 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 12:38:26.112485 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:26.112485 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:26.112485 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 265/518] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:38:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:26.112956 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e71073-4b14-4645-b8f6-f8944b35182d None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:26.174596 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef38d32-235d-482e-a0e0-0a22bf596677 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:26.176078 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef38d32-235d-482e-a0e0-0a22bf596677 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:26.225707 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e71073-4b14-4645-b8f6-f8944b35182d None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:26.265685 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa9d860-9485-4360-af0d-779c95ea1039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:26.266308 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9d860-9485-4360-af0d-779c95ea1039 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:26.266587 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9d860-9485-4360-af0d-779c95ea1039 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:26.266863 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9d860-9485-4360-af0d-779c95ea1039 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:26.269800 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:26.282947 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 254/519] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:38:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 12:38:26.283809 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef38d32-235d-482e-a0e0-0a22bf596677 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:26.292737 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:26.337037 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa9d860-9485-4360-af0d-779c95ea1039 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:26.338520 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa9d860-9485-4360-af0d-779c95ea1039 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:26.382292 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef38d32-235d-482e-a0e0-0a22bf596677 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:26.418993 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0c18e-644e-4435-a7d6-1cdcd49b816c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:26.419629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0c18e-644e-4435-a7d6-1cdcd49b816c None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:26.419945 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0c18e-644e-4435-a7d6-1cdcd49b816c None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:26.420223 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0c18e-644e-4435-a7d6-1cdcd49b816c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:26.427263 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e0c18e-644e-4435-a7d6-1cdcd49b816c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:26.429322 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e0c18e-644e-4435-a7d6-1cdcd49b816c None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:26.439874 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 266/520] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:38:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 12:38:26.440694 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa9d860-9485-4360-af0d-779c95ea1039 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:26.440827 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-c4e0c18e-644e-4435-a7d6-1cdcd49b816c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 12:38:26.440827 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:26.440827 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:26.440827 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:26.440827 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:26.440827 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:26.440827 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:26.440827 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:26.440827 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:26.440827 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:26.440827 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:26.440827 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:26.440827 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 12:38:26.440827 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 12:38:26.440827 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 12:38:26.440827 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 12:38:26.441633 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:26.441633 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:26.441633 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 12:38:26.441633 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:26.441633 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 12:38:26.441633 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:26.441633 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:26.441633 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:26.441633 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:26.441633 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:26.441633 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:26.441633 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:26.441633 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:26.441633 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:26.441633 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:26.441633 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:26.441633 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:26.441633 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:26.442500 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:26.442500 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:26.442500 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 12:38:26.442500 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 12:38:26.442500 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 12:38:26.442500 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 12:38:26.442500 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 12:38:26.442500 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 12:38:26.442500 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 12:38:26.442500 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:26.442500 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 255/521] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 26 12:38:26 2022] GET /identity/v3/users/neutron => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:26.443050 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0c18e-644e-4435-a7d6-1cdcd49b816c None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:26.490986 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:26.496964 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:26.542392 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa9d860-9485-4360-af0d-779c95ea1039 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:26.543321 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0c18e-644e-4435-a7d6-1cdcd49b816c None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:26.577374 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7751c656-a651-4e9b-845e-80f7ce025b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:26.577997 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751c656-a651-4e9b-845e-80f7ce025b00 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:26.578312 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751c656-a651-4e9b-845e-80f7ce025b00 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:26.578546 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751c656-a651-4e9b-845e-80f7ce025b00 None admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:26.579296 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7751c656-a651-4e9b-845e-80f7ce025b00 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:26.579605 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7751c656-a651-4e9b-845e-80f7ce025b00 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:26.580712 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06562fb7-ece3-43cd-99bc-56086f55d555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:26.581078 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7751c656-a651-4e9b-845e-80f7ce025b00 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:26.581336 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06562fb7-ece3-43cd-99bc-56086f55d555 None admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:26.581786 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06562fb7-ece3-43cd-99bc-56086f55d555 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:26.582297 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06562fb7-ece3-43cd-99bc-56086f55d555 None admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:26.583561 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06562fb7-ece3-43cd-99bc-56086f55d555 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:26.586159 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06562fb7-ece3-43cd-99bc-56086f55d555 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:26.602089 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 256/522] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 26 12:38:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:26.603441 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7751c656-a651-4e9b-845e-80f7ce025b00 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:26.612479 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 267/523] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Jul 26 12:38:26 2022] POST /identity/v3/users => generated 247 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:38:26.613860 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06562fb7-ece3-43cd-99bc-56086f55d555 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:26.731560 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7751c656-a651-4e9b-845e-80f7ce025b00 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:26.736627 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06562fb7-ece3-43cd-99bc-56086f55d555 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:26.743194 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:26.751499 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:26.773281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab10e02-02a0-4f53-a0ee-77699029f917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:26.773974 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab10e02-02a0-4f53-a0ee-77699029f917 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:26.774206 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab10e02-02a0-4f53-a0ee-77699029f917 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:26.774472 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab10e02-02a0-4f53-a0ee-77699029f917 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:26.785631 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab10e02-02a0-4f53-a0ee-77699029f917 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:26.787372 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab10e02-02a0-4f53-a0ee-77699029f917 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:26.790180 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c3d913-b87a-4923-b006-012977a65206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:26.790816 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3d913-b87a-4923-b006-012977a65206 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:26.791438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3d913-b87a-4923-b006-012977a65206 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:26.791750 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3d913-b87a-4923-b006-012977a65206 None admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:26.792553 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c3d913-b87a-4923-b006-012977a65206 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:26.792864 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c3d913-b87a-4923-b006-012977a65206 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:26.794368 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c3d913-b87a-4923-b006-012977a65206 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:26.808982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-cab10e02-02a0-4f53-a0ee-77699029f917 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:26.808982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:26.808982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:26.808982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:26.808982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:26.808982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:26.808982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:26.808982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:26.808982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:26.808982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:26.808982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:26.808982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:26.808982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:26.808982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:26.808982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:26.809892 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:26.809892 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:26.809892 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:26.809892 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:26.809892 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:26.809892 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:26.809892 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:26.809892 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:26.809892 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:26.809892 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:26.809892 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:26.809892 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:26.809892 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:26.809892 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:26.809892 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:26.809892 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:26.809892 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:26.810779 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:26.810779 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:26.810779 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:26.810779 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:26.810779 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 257/524] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:26.811364 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab10e02-02a0-4f53-a0ee-77699029f917 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:26.817488 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 268/525] 10.222.0.20 () {56 vars in 1139 bytes} [Tue Jul 26 12:38:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:26.818675 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c3d913-b87a-4923-b006-012977a65206 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:26.923108 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab10e02-02a0-4f53-a0ee-77699029f917 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:26.936163 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c3d913-b87a-4923-b006-012977a65206 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:26.969202 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86490ce4-2d86-4803-bee1-ab4059f04915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:26.969610 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86490ce4-2d86-4803-bee1-ab4059f04915 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:26.969849 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86490ce4-2d86-4803-bee1-ab4059f04915 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:26.970125 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86490ce4-2d86-4803-bee1-ab4059f04915 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:26.981203 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6b9b4e-264d-47a6-90b8-f0c05bafebe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:26.981895 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b9b4e-264d-47a6-90b8-f0c05bafebe2 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:26.982139 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b9b4e-264d-47a6-90b8-f0c05bafebe2 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:26.982361 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86490ce4-2d86-4803-bee1-ab4059f04915 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:26.982495 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b9b4e-264d-47a6-90b8-f0c05bafebe2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:26.983168 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6b9b4e-264d-47a6-90b8-f0c05bafebe2 None admin] RBAC: Adding query filter params (role.id=9c68629183974be9bfb1ed95f69912ee, scope.project.id=644278e7e8934b37a1e7ab20d35764cd, user.id=0f5d6f3c12244c0faf26048513dc49e7) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:26.983480 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6b9b4e-264d-47a6-90b8-f0c05bafebe2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:26.983870 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86490ce4-2d86-4803-bee1-ab4059f04915 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:26.984942 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6b9b4e-264d-47a6-90b8-f0c05bafebe2 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:26.991147 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 269/526] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:38:26 2022] GET /identity/v3/role_assignments?user.id=0f5d6f3c12244c0faf26048513dc49e7&scope.project.id=644278e7e8934b37a1e7ab20d35764cd&role.id=9c68629183974be9bfb1ed95f69912ee => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:26.991982 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6b9b4e-264d-47a6-90b8-f0c05bafebe2 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:27.012252 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-86490ce4-2d86-4803-bee1-ab4059f04915 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:27.012252 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:27.012252 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:27.012252 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:27.012252 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:27.012252 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:27.012252 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:27.012252 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:27.012252 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:27.012252 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:27.012252 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:27.012252 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:27.012252 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:27.012252 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:27.012252 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:27.013326 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:27.013326 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:27.013326 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:27.013326 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:27.013326 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:27.013326 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:27.013326 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:27.013326 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:27.013326 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:27.013326 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:27.013326 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:27.013326 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:27.013326 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:27.013326 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:27.013326 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:27.013326 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:27.013326 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:27.014388 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:27.014388 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:27.014388 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:27.014388 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:27.016490 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 258/527] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:27.017800 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86490ce4-2d86-4803-bee1-ab4059f04915 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:27.038865 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:27.046524 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:27.144782 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6b9b4e-264d-47a6-90b8-f0c05bafebe2 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:27.158016 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86490ce4-2d86-4803-bee1-ab4059f04915 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:27.186978 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac1e5d-5e18-4cf1-9306-b45ff725d3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:27.187607 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac1e5d-5e18-4cf1-9306-b45ff725d3f8 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:27.187921 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac1e5d-5e18-4cf1-9306-b45ff725d3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:27.188197 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac1e5d-5e18-4cf1-9306-b45ff725d3f8 None admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:27.188898 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ac1e5d-5e18-4cf1-9306-b45ff725d3f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:27.189201 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ac1e5d-5e18-4cf1-9306-b45ff725d3f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:27.190600 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ac1e5d-5e18-4cf1-9306-b45ff725d3f8 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:27.203574 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 270/528] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:27.204184 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a60026-b49c-4fef-aee1-b5b29331565f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:27.204438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac1e5d-5e18-4cf1-9306-b45ff725d3f8 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:27.204872 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a60026-b49c-4fef-aee1-b5b29331565f None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:27.205182 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a60026-b49c-4fef-aee1-b5b29331565f None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:27.205456 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a60026-b49c-4fef-aee1-b5b29331565f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:27.301494 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a60026-b49c-4fef-aee1-b5b29331565f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:27.304170 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a60026-b49c-4fef-aee1-b5b29331565f None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:27.313223 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:27.324897 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac1e5d-5e18-4cf1-9306-b45ff725d3f8 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:27.325470 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:27.362389 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06aab88-4859-48f2-8e48-d4a239508df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:27.363040 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06aab88-4859-48f2-8e48-d4a239508df5 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:27.363324 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06aab88-4859-48f2-8e48-d4a239508df5 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:27.363598 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06aab88-4859-48f2-8e48-d4a239508df5 None admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:27.364604 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06aab88-4859-48f2-8e48-d4a239508df5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:27.364913 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06aab88-4859-48f2-8e48-d4a239508df5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:27.366842 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06aab88-4859-48f2-8e48-d4a239508df5 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:27.379784 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 271/529] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:27.380720 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06aab88-4859-48f2-8e48-d4a239508df5 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:27.439731 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 259/530] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:38:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 12:38:27.440692 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a60026-b49c-4fef-aee1-b5b29331565f None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:27.514853 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06aab88-4859-48f2-8e48-d4a239508df5 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:27.552389 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d0351b-3589-4932-9bb8-407f043a3bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:27.553511 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0351b-3589-4932-9bb8-407f043a3bca None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:27.553909 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0351b-3589-4932-9bb8-407f043a3bca None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:27.554186 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0351b-3589-4932-9bb8-407f043a3bca None admin] PATH_INFO: `/v3/projects/service` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:27.555036 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a60026-b49c-4fef-aee1-b5b29331565f None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:27.561701 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d0351b-3589-4932-9bb8-407f043a3bca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:27.563184 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d0351b-3589-4932-9bb8-407f043a3bca None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:27.571626 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:27.581186 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-10d0351b-3589-4932-9bb8-407f043a3bca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:27.581186 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:27.581186 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:27.581186 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:27.581186 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:27.581186 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:27.581186 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:27.581186 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:27.581186 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:27.581186 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:27.581186 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:27.581186 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:27.581186 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 12:38:27.581186 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 12:38:27.581186 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 12:38:27.582306 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 12:38:27.582306 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:27.582306 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:27.582306 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:27.582306 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:27.582306 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:27.582306 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:27.582306 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:27.582306 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:27.582306 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:27.582306 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:27.582306 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:27.582306 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:27.582306 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:27.582306 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:27.582306 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:27.582306 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:27.583144 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 12:38:27.583144 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 12:38:27.583144 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 12:38:27.583144 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 12:38:27.583144 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 12:38:27.583144 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 12:38:27.583144 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:27.583144 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:27.583144 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 272/531] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:38:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:27.583144 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d0351b-3589-4932-9bb8-407f043a3bca None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:27.599163 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c33a8-2766-4cc2-82d3-b138fa3ad226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:27.600539 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:27.600719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c33a8-2766-4cc2-82d3-b138fa3ad226 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:27.600938 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c33a8-2766-4cc2-82d3-b138fa3ad226 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:27.601373 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c33a8-2766-4cc2-82d3-b138fa3ad226 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:27.613143 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083c33a8-2766-4cc2-82d3-b138fa3ad226 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:27.615817 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083c33a8-2766-4cc2-82d3-b138fa3ad226 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:27.632933 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-083c33a8-2766-4cc2-82d3-b138fa3ad226 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:27.632933 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:27.632933 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:27.632933 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:27.632933 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:27.632933 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:27.632933 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:27.632933 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:27.632933 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:27.632933 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:27.632933 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:27.632933 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:27.632933 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 12:38:27.632933 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 12:38:27.632933 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 12:38:27.633831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 12:38:27.633831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:27.633831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:27.633831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:27.633831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:27.633831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:27.633831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:27.633831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:27.633831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:27.633831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:27.633831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:27.633831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:27.633831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:27.633831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:27.633831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:27.633831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:27.633831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:27.634662 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 12:38:27.634662 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 12:38:27.634662 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 12:38:27.634662 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 12:38:27.634662 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 12:38:27.634662 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 12:38:27.634662 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:27.634662 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:27.637334 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 260/532] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:38:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:27.638833 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c33a8-2766-4cc2-82d3-b138fa3ad226 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:27.706738 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d0351b-3589-4932-9bb8-407f043a3bca None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:27.743439 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b643e3-3ebf-412e-8952-c1f00c744008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:27.744221 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b643e3-3ebf-412e-8952-c1f00c744008 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:27.744501 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b643e3-3ebf-412e-8952-c1f00c744008 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:27.744771 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b643e3-3ebf-412e-8952-c1f00c744008 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:27.753358 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c33a8-2766-4cc2-82d3-b138fa3ad226 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:27.756937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b643e3-3ebf-412e-8952-c1f00c744008 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:27.758497 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b643e3-3ebf-412e-8952-c1f00c744008 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:27.774354 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-30b643e3-3ebf-412e-8952-c1f00c744008 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:27.774354 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:27.774354 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:27.774354 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:27.774354 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:27.774354 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:27.774354 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:27.774354 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:27.774354 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:27.774354 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:27.774354 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:27.774354 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:27.774354 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:27.774354 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:27.774354 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:27.775262 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:27.775262 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:27.775262 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:27.775262 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:27.775262 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:27.775262 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:27.775262 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:27.775262 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:27.775262 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:27.775262 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:27.775262 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:27.775262 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:27.775262 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:27.775262 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:27.775262 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:27.775262 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:27.775262 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:27.776784 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:27.776784 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:27.776784 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:27.776784 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:27.776784 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 273/533] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:27.777067 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b643e3-3ebf-412e-8952-c1f00c744008 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:27.792141 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2470cd55-eff7-467d-9b8b-618451f53081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:27.792778 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2470cd55-eff7-467d-9b8b-618451f53081 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:27.793062 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2470cd55-eff7-467d-9b8b-618451f53081 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:27.793442 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2470cd55-eff7-467d-9b8b-618451f53081 None admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:27.794363 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2470cd55-eff7-467d-9b8b-618451f53081 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:27.794681 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2470cd55-eff7-467d-9b8b-618451f53081 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:27.796371 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2470cd55-eff7-467d-9b8b-618451f53081 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:27.812202 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 261/534] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 26 12:38:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:27.813208 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2470cd55-eff7-467d-9b8b-618451f53081 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:27.845744 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:27.848941 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:27.899193 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b643e3-3ebf-412e-8952-c1f00c744008 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:27.938694 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2470cd55-eff7-467d-9b8b-618451f53081 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:27.940988 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf4f18-7d8f-423c-803d-0f4767267f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:27.941772 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf4f18-7d8f-423c-803d-0f4767267f4d None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:27.942148 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf4f18-7d8f-423c-803d-0f4767267f4d None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:27.942515 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf4f18-7d8f-423c-803d-0f4767267f4d None admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:27.943374 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf4f18-7d8f-423c-803d-0f4767267f4d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:27.944315 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf4f18-7d8f-423c-803d-0f4767267f4d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:27.945837 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf4f18-7d8f-423c-803d-0f4767267f4d None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:27.970480 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 274/535] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 26 12:38:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:27.971592 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf4f18-7d8f-423c-803d-0f4767267f4d None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:27.976121 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb29d32-0628-4ada-8ee3-020c50550980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:27.976842 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb29d32-0628-4ada-8ee3-020c50550980 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:27.977128 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb29d32-0628-4ada-8ee3-020c50550980 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:27.977405 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb29d32-0628-4ada-8ee3-020c50550980 None admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:27.978101 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb29d32-0628-4ada-8ee3-020c50550980 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:27.978408 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb29d32-0628-4ada-8ee3-020c50550980 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:27.979821 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb29d32-0628-4ada-8ee3-020c50550980 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:27.993214 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 262/536] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:27.994155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb29d32-0628-4ada-8ee3-020c50550980 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:28.078612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf4f18-7d8f-423c-803d-0f4767267f4d None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:28.093406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:28.104587 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb29d32-0628-4ada-8ee3-020c50550980 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:28.106116 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:28.118224 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4463f6-81d5-4022-8d3b-dcbc15b5cb4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:28.118997 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4463f6-81d5-4022-8d3b-dcbc15b5cb4e None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:28.119375 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4463f6-81d5-4022-8d3b-dcbc15b5cb4e None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:28.119774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4463f6-81d5-4022-8d3b-dcbc15b5cb4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:28.120659 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4463f6-81d5-4022-8d3b-dcbc15b5cb4e None admin] RBAC: Adding query filter params (role.id=9c68629183974be9bfb1ed95f69912ee, scope.project.id=644278e7e8934b37a1e7ab20d35764cd, user.id=d2b5853838dc40db89c46038c0606107) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:28.121137 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4463f6-81d5-4022-8d3b-dcbc15b5cb4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:28.122760 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4463f6-81d5-4022-8d3b-dcbc15b5cb4e None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:28.129219 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 275/537] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:38:27 2022] GET /identity/v3/role_assignments?user.id=d2b5853838dc40db89c46038c0606107&scope.project.id=644278e7e8934b37a1e7ab20d35764cd&role.id=9c68629183974be9bfb1ed95f69912ee => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:28.130208 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4463f6-81d5-4022-8d3b-dcbc15b5cb4e None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:28.144682 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12df0e9-8c3c-4789-85d7-4a44644dab75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:28.145319 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12df0e9-8c3c-4789-85d7-4a44644dab75 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:28.145599 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12df0e9-8c3c-4789-85d7-4a44644dab75 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:28.145871 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12df0e9-8c3c-4789-85d7-4a44644dab75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:28.146662 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12df0e9-8c3c-4789-85d7-4a44644dab75 None admin] RBAC: Adding query filter params (role.id=9c68629183974be9bfb1ed95f69912ee, scope.project.id=644278e7e8934b37a1e7ab20d35764cd, user.id=9f8ca5e019dd4b9090cee2b2c1d8e15d) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:28.147619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12df0e9-8c3c-4789-85d7-4a44644dab75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:28.149196 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12df0e9-8c3c-4789-85d7-4a44644dab75 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:28.155729 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 263/538] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:38:27 2022] GET /identity/v3/role_assignments?user.id=9f8ca5e019dd4b9090cee2b2c1d8e15d&scope.project.id=644278e7e8934b37a1e7ab20d35764cd&role.id=9c68629183974be9bfb1ed95f69912ee => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:28.295462 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4463f6-81d5-4022-8d3b-dcbc15b5cb4e None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:28.341067 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e4641-69ec-4cec-a090-1734e44e9e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:28.342025 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e4641-69ec-4cec-a090-1734e44e9e1f None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:28.342538 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e4641-69ec-4cec-a090-1734e44e9e1f None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:28.343011 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e4641-69ec-4cec-a090-1734e44e9e1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:28.351217 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e4641-69ec-4cec-a090-1734e44e9e1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:28.353054 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e4641-69ec-4cec-a090-1734e44e9e1f None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:28.368049 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-5c4e4641-69ec-4cec-a090-1734e44e9e1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:28.368049 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:28.368049 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:28.368049 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:28.368049 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:28.368049 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:28.368049 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:28.368049 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:28.368049 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:28.368049 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:28.368049 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:28.368049 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:28.368049 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 12:38:28.368049 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 12:38:28.368049 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 12:38:28.368981 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 12:38:28.368981 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:28.368981 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:28.368981 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:28.368981 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:28.368981 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:28.368981 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:28.368981 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:28.368981 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:28.368981 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:28.368981 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:28.368981 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:28.368981 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:28.368981 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:28.368981 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:28.368981 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:28.368981 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:28.369806 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 12:38:28.369806 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 12:38:28.369806 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 12:38:28.369806 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 12:38:28.369806 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 12:38:28.369806 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 12:38:28.369806 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:28.369806 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:28.372097 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 276/539] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:38:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:28.379841 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12df0e9-8c3c-4789-85d7-4a44644dab75 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:28.541347 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12df0e9-8c3c-4789-85d7-4a44644dab75 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:28.581429 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850ce44a-5da5-475f-a93b-cfdea9b91638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:28.582413 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ce44a-5da5-475f-a93b-cfdea9b91638 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:28.582936 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ce44a-5da5-475f-a93b-cfdea9b91638 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:28.583410 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ce44a-5da5-475f-a93b-cfdea9b91638 None admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:28.584518 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850ce44a-5da5-475f-a93b-cfdea9b91638 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:28.585033 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850ce44a-5da5-475f-a93b-cfdea9b91638 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:28.586675 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850ce44a-5da5-475f-a93b-cfdea9b91638 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:28.605883 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 264/540] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 26 12:38:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:28.611816 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e4641-69ec-4cec-a090-1734e44e9e1f None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:28.692311 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:28.732091 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e4641-69ec-4cec-a090-1734e44e9e1f None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:28.767261 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4703e33f-d6dd-4e60-b596-ecabc78a05bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:28.767900 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703e33f-d6dd-4e60-b596-ecabc78a05bb None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:28.768177 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703e33f-d6dd-4e60-b596-ecabc78a05bb None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:28.768447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703e33f-d6dd-4e60-b596-ecabc78a05bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:28.769227 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4703e33f-d6dd-4e60-b596-ecabc78a05bb None admin] RBAC: Adding query filter params (role.id=9c68629183974be9bfb1ed95f69912ee, scope.project.id=644278e7e8934b37a1e7ab20d35764cd, user.id=6830350c6eba4610bbe9ab1dc31f5939) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:28.769532 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4703e33f-d6dd-4e60-b596-ecabc78a05bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:28.770983 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4703e33f-d6dd-4e60-b596-ecabc78a05bb None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:28.777623 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 277/541] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:38:28 2022] GET /identity/v3/role_assignments?user.id=6830350c6eba4610bbe9ab1dc31f5939&scope.project.id=644278e7e8934b37a1e7ab20d35764cd&role.id=9c68629183974be9bfb1ed95f69912ee => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:28.909422 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8601078-780b-47a4-84f6-976bd3d194f1 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:28.910061 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8601078-780b-47a4-84f6-976bd3d194f1 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:28.910553 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8601078-780b-47a4-84f6-976bd3d194f1 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:28.913146 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 265/542] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 12:38:28.919323 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57fda1-a987-4940-bd1f-8af5b8e7c24e None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:28.919993 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57fda1-a987-4940-bd1f-8af5b8e7c24e None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:28.920494 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57fda1-a987-4940-bd1f-8af5b8e7c24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:28.944181 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-2d57fda1-a987-4940-bd1f-8af5b8e7c24e None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:28.946218 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-2d57fda1-a987-4940-bd1f-8af5b8e7c24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:29.050768 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 278/543] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:38:29.083786 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8601078-780b-47a4-84f6-976bd3d194f1 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:29.105768 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:29.217273 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8601078-780b-47a4-84f6-976bd3d194f1 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:29.273653 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c794cf9d-c02c-4643-b226-ca1e9314f049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:29.274865 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794cf9d-c02c-4643-b226-ca1e9314f049 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:29.275148 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794cf9d-c02c-4643-b226-ca1e9314f049 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:29.275417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794cf9d-c02c-4643-b226-ca1e9314f049 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:29.293185 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c794cf9d-c02c-4643-b226-ca1e9314f049 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:38:29.298096 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-c794cf9d-c02c-4643-b226-ca1e9314f049 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 12:38:29.298096 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:29.298096 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:29.298096 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:29.298096 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:29.298096 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:29.298096 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:29.298096 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:29.298096 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:29.298096 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:29.298096 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:29.298096 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:29.298096 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:38:29.298096 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:38:29.298096 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:38:29.298096 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:38:29.298963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:38:29.298963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:38:29.298963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:38:29.298963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:38:29.298963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:29.298963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:29.298963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:29.298963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:29.298963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:29.298963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:29.298963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:29.298963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:29.298963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:29.298963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:29.298963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:29.298963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:29.300155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:29.300155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:29.300155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:29.300155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:29.300155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:38:29.300155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:38:29.300155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:38:29.300155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:38:29.300155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:38:29.300155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:38:29.300155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 12:38:29.300155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:29.300155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 266/544] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 26 12:38:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:29.305216 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57fda1-a987-4940-bd1f-8af5b8e7c24e None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:29.366773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b97f8-f1a7-4e3e-85d2-0ff77413b007 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:29.367117 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b97f8-f1a7-4e3e-85d2-0ff77413b007 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:29.367331 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b97f8-f1a7-4e3e-85d2-0ff77413b007 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:29.368972 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 267/545] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 12:38:29.374005 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd849e-3e2f-4c97-984c-d394b9046455 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:29.374288 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd849e-3e2f-4c97-984c-d394b9046455 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:29.374562 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd849e-3e2f-4c97-984c-d394b9046455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:29.379978 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:29.394569 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-12bd849e-3e2f-4c97-984c-d394b9046455 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:29.396165 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-12bd849e-3e2f-4c97-984c-d394b9046455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:29.471975 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57fda1-a987-4940-bd1f-8af5b8e7c24e None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:29.502161 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 268/546] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 12:38:29.532744 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ce7f1-3907-4c45-bd81-e68a4ae1d59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:29.533292 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bd849e-3e2f-4c97-984c-d394b9046455 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:29.533427 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ce7f1-3907-4c45-bd81-e68a4ae1d59e None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:29.533632 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ce7f1-3907-4c45-bd81-e68a4ae1d59e None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:29.533909 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ce7f1-3907-4c45-bd81-e68a4ae1d59e None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:29.534637 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9ce7f1-3907-4c45-bd81-e68a4ae1d59e None admin] RBAC: Adding query filter params (name=service) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:29.534947 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9ce7f1-3907-4c45-bd81-e68a4ae1d59e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:29.536427 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9ce7f1-3907-4c45-bd81-e68a4ae1d59e None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:29.544354 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:29.559325 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 279/547] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Jul 26 12:38:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:29.562890 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ce7f1-3907-4c45-bd81-e68a4ae1d59e None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:29.679663 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bd849e-3e2f-4c97-984c-d394b9046455 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:29.714363 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca786c98-5fc8-40dd-9642-dd8c509b9ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:29.715005 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca786c98-5fc8-40dd-9642-dd8c509b9ec0 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:29.715282 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca786c98-5fc8-40dd-9642-dd8c509b9ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:29.715550 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca786c98-5fc8-40dd-9642-dd8c509b9ec0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:29.737355 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca786c98-5fc8-40dd-9642-dd8c509b9ec0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:38:29.743819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-ca786c98-5fc8-40dd-9642-dd8c509b9ec0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 12:38:29.743819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:29.743819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:29.743819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:29.743819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:29.743819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:29.743819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:29.743819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:29.743819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:29.743819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:29.743819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:29.743819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:29.743819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:38:29.743819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:38:29.743819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:38:29.743819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:38:29.744681 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:38:29.744681 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:38:29.744681 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:38:29.744681 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:38:29.744681 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:29.744681 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:29.744681 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:29.744681 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:29.744681 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:29.744681 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:29.744681 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:29.744681 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:29.744681 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:29.744681 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:29.744681 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:29.744681 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:29.745460 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:29.745460 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:29.745460 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:29.745460 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:29.745460 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:38:29.745460 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:38:29.745460 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:38:29.745460 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:38:29.745460 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:38:29.745460 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:38:29.745460 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 12:38:29.745460 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:29.747141 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 269/548] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 26 12:38:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:38:29.756420 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca786c98-5fc8-40dd-9642-dd8c509b9ec0 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:29.790070 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ce7f1-3907-4c45-bd81-e68a4ae1d59e None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:29.796494 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:29.858174 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86811cca-2066-41c4-8503-be458d8e6baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:29.858827 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86811cca-2066-41c4-8503-be458d8e6baf None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:29.859102 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86811cca-2066-41c4-8503-be458d8e6baf None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:29.859370 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86811cca-2066-41c4-8503-be458d8e6baf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:29.886371 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86811cca-2066-41c4-8503-be458d8e6baf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:29.890107 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86811cca-2066-41c4-8503-be458d8e6baf None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:29.890861 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:29.918097 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-86811cca-2066-41c4-8503-be458d8e6baf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:29.918097 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:29.918097 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:29.918097 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:29.918097 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:29.918097 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:29.918097 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:29.918097 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:29.918097 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:29.918097 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:29.918097 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:29.918097 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:29.918097 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:29.918097 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:29.918097 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:29.919416 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:29.919416 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:29.919416 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:29.919416 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:29.919416 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:29.919416 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:29.919416 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:29.919416 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:29.919416 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:29.919416 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:29.919416 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:29.919416 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:29.919416 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:29.919416 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:29.919416 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:29.919416 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:29.919416 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:29.920525 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:29.920525 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:29.920525 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:29.920525 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:29.924013 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 280/549] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:38:29.924906 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86811cca-2066-41c4-8503-be458d8e6baf None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:29.935846 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca786c98-5fc8-40dd-9642-dd8c509b9ec0 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:29.972092 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0d5498-2c18-4ef2-a7a1-6720f2c35c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:29.972719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d5498-2c18-4ef2-a7a1-6720f2c35c88 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:29.973008 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d5498-2c18-4ef2-a7a1-6720f2c35c88 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:29.973289 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d5498-2c18-4ef2-a7a1-6720f2c35c88 None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:29.974014 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0d5498-2c18-4ef2-a7a1-6720f2c35c88 None admin] RBAC: Adding query filter params (name=service) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:29.974324 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0d5498-2c18-4ef2-a7a1-6720f2c35c88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:29.976255 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0d5498-2c18-4ef2-a7a1-6720f2c35c88 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:29.984782 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 270/550] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Jul 26 12:38:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:29.991107 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0d5498-2c18-4ef2-a7a1-6720f2c35c88 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:30.129508 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86811cca-2066-41c4-8503-be458d8e6baf None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:30.173295 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0d5498-2c18-4ef2-a7a1-6720f2c35c88 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:30.173831 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd5a470-37fa-46d5-8834-b29843518955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:30.179224 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5a470-37fa-46d5-8834-b29843518955 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:30.179508 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5a470-37fa-46d5-8834-b29843518955 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:30.179807 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5a470-37fa-46d5-8834-b29843518955 None admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:30.180583 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd5a470-37fa-46d5-8834-b29843518955 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:30.180888 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd5a470-37fa-46d5-8834-b29843518955 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:30.183226 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd5a470-37fa-46d5-8834-b29843518955 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:30.198237 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 281/551] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:30.203550 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd5a470-37fa-46d5-8834-b29843518955 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:30.222604 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:30.250740 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:30.271823 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b52c0b-899f-41c1-94e3-2edfbc711f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:30.272478 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b52c0b-899f-41c1-94e3-2edfbc711f7e None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:30.272764 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b52c0b-899f-41c1-94e3-2edfbc711f7e None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:30.273041 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b52c0b-899f-41c1-94e3-2edfbc711f7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:30.308862 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b52c0b-899f-41c1-94e3-2edfbc711f7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:30.310363 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b52c0b-899f-41c1-94e3-2edfbc711f7e None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:30.312268 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd5a470-37fa-46d5-8834-b29843518955 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:30.339254 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-17b52c0b-899f-41c1-94e3-2edfbc711f7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:30.339254 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:30.339254 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:30.339254 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:30.339254 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:30.339254 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:30.339254 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:30.339254 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:30.339254 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:30.339254 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:30.339254 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:30.339254 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:30.339254 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:30.339254 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:30.339254 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:30.340299 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:30.340299 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:30.340299 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:30.340299 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:30.340299 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:30.340299 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:30.340299 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:30.340299 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:30.340299 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:30.340299 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:30.340299 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:30.340299 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:30.340299 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:30.340299 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:30.340299 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:30.340299 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:30.340299 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:30.341254 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:30.341254 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:30.341254 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:30.341254 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:30.347223 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 271/552] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:38:30.348451 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b52c0b-899f-41c1-94e3-2edfbc711f7e None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:30.385984 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23728a83-5050-4e95-84fa-47b3db6e38f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:30.386627 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23728a83-5050-4e95-84fa-47b3db6e38f2 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:30.386911 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23728a83-5050-4e95-84fa-47b3db6e38f2 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:30.387181 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23728a83-5050-4e95-84fa-47b3db6e38f2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:30.401158 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23728a83-5050-4e95-84fa-47b3db6e38f2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:30.402635 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23728a83-5050-4e95-84fa-47b3db6e38f2 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:30.426612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-23728a83-5050-4e95-84fa-47b3db6e38f2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 12:38:30.426612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:30.426612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:30.426612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:30.426612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:30.426612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:30.426612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:30.426612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:30.426612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:30.426612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:30.426612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:30.426612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:30.426612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 12:38:30.426612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 12:38:30.426612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 12:38:30.426612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 12:38:30.427577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:30.427577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:30.427577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 12:38:30.427577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:30.427577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 12:38:30.427577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:30.427577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:30.427577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:30.427577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:30.427577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:30.427577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:30.427577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:30.427577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:30.427577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:30.427577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:30.427577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:30.427577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:30.427577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:30.428594 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:30.428594 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:30.428594 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 12:38:30.428594 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 12:38:30.428594 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 12:38:30.428594 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 12:38:30.428594 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 12:38:30.428594 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 12:38:30.428594 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 12:38:30.428594 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:30.428594 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 282/553] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 26 12:38:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:38:30.436748 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23728a83-5050-4e95-84fa-47b3db6e38f2 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:30.520083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:30.548004 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23728a83-5050-4e95-84fa-47b3db6e38f2 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:30.551581 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b52c0b-899f-41c1-94e3-2edfbc711f7e None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:30.583195 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec64b9-2c50-49f7-90ab-2a77baad7a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:30.583992 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec64b9-2c50-49f7-90ab-2a77baad7a7b None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:30.584888 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec64b9-2c50-49f7-90ab-2a77baad7a7b None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:30.585254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec64b9-2c50-49f7-90ab-2a77baad7a7b None admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:30.586137 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbec64b9-2c50-49f7-90ab-2a77baad7a7b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:30.586535 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbec64b9-2c50-49f7-90ab-2a77baad7a7b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:30.588099 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbec64b9-2c50-49f7-90ab-2a77baad7a7b None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:30.593240 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d2e32a-5709-4b56-a48f-29f95bc2fd50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:30.593867 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d2e32a-5709-4b56-a48f-29f95bc2fd50 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:30.594191 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d2e32a-5709-4b56-a48f-29f95bc2fd50 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:30.594469 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d2e32a-5709-4b56-a48f-29f95bc2fd50 None admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:30.595318 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d2e32a-5709-4b56-a48f-29f95bc2fd50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:30.595808 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d2e32a-5709-4b56-a48f-29f95bc2fd50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:30.597431 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d2e32a-5709-4b56-a48f-29f95bc2fd50 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:30.607567 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 283/554] 10.222.0.20 () {56 vars in 1135 bytes} [Tue Jul 26 12:38:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:30.609032 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e36803e-2252-4e43-a997-6547de9aaacb None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:30.609384 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e36803e-2252-4e43-a997-6547de9aaacb None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:30.609747 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e36803e-2252-4e43-a997-6547de9aaacb None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:30.611265 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 284/555] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:30.612636 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 272/556] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:30.618133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641c6f8-484f-4a75-b6d7-5247726924ef None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:30.618418 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641c6f8-484f-4a75-b6d7-5247726924ef None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:30.618695 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641c6f8-484f-4a75-b6d7-5247726924ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:30.620245 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d2e32a-5709-4b56-a48f-29f95bc2fd50 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:30.642549 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-a641c6f8-484f-4a75-b6d7-5247726924ef None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:30.644401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-a641c6f8-484f-4a75-b6d7-5247726924ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:30.670021 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:30.735571 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d2e32a-5709-4b56-a48f-29f95bc2fd50 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:30.736509 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 285/557] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:38:30.737722 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a641c6f8-484f-4a75-b6d7-5247726924ef None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:30.786832 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303b03f7-f45d-4913-96b1-841ff4a99ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:30.787908 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b03f7-f45d-4913-96b1-841ff4a99ce7 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:30.788188 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b03f7-f45d-4913-96b1-841ff4a99ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:30.788452 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b03f7-f45d-4913-96b1-841ff4a99ce7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:30.816487 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:30.858225 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a641c6f8-484f-4a75-b6d7-5247726924ef None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:30.861013 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303b03f7-f45d-4913-96b1-841ff4a99ce7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:30.862470 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303b03f7-f45d-4913-96b1-841ff4a99ce7 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:30.902010 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec7c929-b754-4eba-bcf0-30f673af6d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:30.902657 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec7c929-b754-4eba-bcf0-30f673af6d65 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:30.902940 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec7c929-b754-4eba-bcf0-30f673af6d65 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:30.903219 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec7c929-b754-4eba-bcf0-30f673af6d65 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:30.908876 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:30.913790 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec7c929-b754-4eba-bcf0-30f673af6d65 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:30.915143 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec7c929-b754-4eba-bcf0-30f673af6d65 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:30.928825 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-3ec7c929-b754-4eba-bcf0-30f673af6d65 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 12:38:30.928825 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:30.928825 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:30.928825 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:30.928825 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:30.928825 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:30.928825 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:30.928825 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:30.928825 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:30.928825 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:30.928825 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:30.928825 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:30.928825 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 12:38:30.928825 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 12:38:30.928825 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 12:38:30.928825 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 12:38:30.930254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:30.930254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:30.930254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 12:38:30.930254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:30.930254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 12:38:30.930254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:30.930254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:30.930254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:30.930254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:30.930254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:30.930254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:30.930254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:30.930254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:30.930254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:30.930254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:30.930254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:30.930254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:30.930254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:30.931826 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:30.931826 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:30.931826 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 12:38:30.931826 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 12:38:30.931826 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 12:38:30.931826 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 12:38:30.931826 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 12:38:30.931826 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 12:38:30.931826 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 12:38:30.931826 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:30.931826 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 286/558] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Jul 26 12:38:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:30.931826 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10fe7b-42cc-428c-a879-2e381604b81d None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:30.931826 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10fe7b-42cc-428c-a879-2e381604b81d None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:30.932589 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10fe7b-42cc-428c-a879-2e381604b81d None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:30.933263 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 287/559] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:30.933914 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10fe7b-42cc-428c-a879-2e381604b81d None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:30.983282 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 273/560] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:38:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 12:38:30.983980 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303b03f7-f45d-4913-96b1-841ff4a99ce7 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:31.052483 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10fe7b-42cc-428c-a879-2e381604b81d None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:31.079277 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:31.087203 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303b03f7-f45d-4913-96b1-841ff4a99ce7 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:31.091064 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199fbb90-3b92-41b6-a493-bb8034cfcd72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:31.091742 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199fbb90-3b92-41b6-a493-bb8034cfcd72 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:31.092028 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199fbb90-3b92-41b6-a493-bb8034cfcd72 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:31.092306 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199fbb90-3b92-41b6-a493-bb8034cfcd72 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:31.104401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199fbb90-3b92-41b6-a493-bb8034cfcd72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:38:31.109689 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-199fbb90-3b92-41b6-a493-bb8034cfcd72 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 12:38:31.109689 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:31.109689 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:31.109689 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:31.109689 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:31.109689 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:31.109689 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:31.109689 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:31.109689 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:31.109689 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:31.109689 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:31.109689 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:31.109689 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:38:31.109689 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:38:31.109689 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:38:31.109689 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:38:31.110939 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:38:31.110939 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:38:31.110939 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:38:31.110939 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:38:31.110939 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:31.110939 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:31.110939 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:31.110939 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:31.110939 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:31.110939 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:31.110939 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:31.110939 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:31.110939 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:31.110939 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:31.110939 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:31.110939 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:31.111783 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:31.111783 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:31.111783 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:31.111783 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:31.111783 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:38:31.111783 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:38:31.111783 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:38:31.111783 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:38:31.111783 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:38:31.111783 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:38:31.111783 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 12:38:31.111783 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:31.111783 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 288/561] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 26 12:38:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:31.112472 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4cf2f2-181e-48db-a5c3-9d12a8b08fb2 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:31.112472 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4cf2f2-181e-48db-a5c3-9d12a8b08fb2 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:31.112605 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4cf2f2-181e-48db-a5c3-9d12a8b08fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:31.130073 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e999422e-69fb-4c76-87f8-5ba0cc808ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:31.130691 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999422e-69fb-4c76-87f8-5ba0cc808ac7 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:31.130967 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999422e-69fb-4c76-87f8-5ba0cc808ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:31.132214 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999422e-69fb-4c76-87f8-5ba0cc808ac7 None admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:31.132460 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-3b4cf2f2-181e-48db-a5c3-9d12a8b08fb2 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:31.132908 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e999422e-69fb-4c76-87f8-5ba0cc808ac7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:31.133277 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e999422e-69fb-4c76-87f8-5ba0cc808ac7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:31.133706 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-3b4cf2f2-181e-48db-a5c3-9d12a8b08fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:31.134704 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e999422e-69fb-4c76-87f8-5ba0cc808ac7 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:31.145972 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:31.155563 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 274/562] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Jul 26 12:38:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:31.156327 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e999422e-69fb-4c76-87f8-5ba0cc808ac7 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:31.219677 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 289/563] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:38:31.219677 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4cf2f2-181e-48db-a5c3-9d12a8b08fb2 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:31.274061 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e999422e-69fb-4c76-87f8-5ba0cc808ac7 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:31.309554 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cfdb3f-4d9d-4ff1-b12d-9529b7ef764f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:31.310109 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cfdb3f-4d9d-4ff1-b12d-9529b7ef764f None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:31.310385 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cfdb3f-4d9d-4ff1-b12d-9529b7ef764f None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:31.310655 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cfdb3f-4d9d-4ff1-b12d-9529b7ef764f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:31.321559 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cfdb3f-4d9d-4ff1-b12d-9529b7ef764f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:31.323106 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cfdb3f-4d9d-4ff1-b12d-9529b7ef764f None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:31.328337 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4cf2f2-181e-48db-a5c3-9d12a8b08fb2 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:31.337968 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-f8cfdb3f-4d9d-4ff1-b12d-9529b7ef764f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:31.337968 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:31.337968 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:31.337968 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:31.337968 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:31.337968 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:31.337968 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:31.337968 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:31.337968 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:31.337968 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:31.337968 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:31.337968 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:31.337968 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:31.337968 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:31.337968 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:31.338863 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:31.338863 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:31.338863 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:31.338863 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:31.338863 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:31.338863 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:31.338863 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:31.338863 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:31.338863 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:31.338863 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:31.338863 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:31.338863 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:31.338863 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:31.338863 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:31.338863 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:31.338863 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:31.338863 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:31.339966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:31.339966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:31.339966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:31.339966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:31.339966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 275/564] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:31.339966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cfdb3f-4d9d-4ff1-b12d-9529b7ef764f None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:31.343904 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:31.364195 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba989f4-b84b-4f3d-8a14-30c54c2100c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:31.365064 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba989f4-b84b-4f3d-8a14-30c54c2100c7 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:31.365341 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba989f4-b84b-4f3d-8a14-30c54c2100c7 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:31.366118 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba989f4-b84b-4f3d-8a14-30c54c2100c7 None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:31.366821 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba989f4-b84b-4f3d-8a14-30c54c2100c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:31.367119 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba989f4-b84b-4f3d-8a14-30c54c2100c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:31.368645 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba989f4-b84b-4f3d-8a14-30c54c2100c7 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:31.376891 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 290/565] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Jul 26 12:38:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:31.377663 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba989f4-b84b-4f3d-8a14-30c54c2100c7 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:31.390817 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:31.441345 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cfdb3f-4d9d-4ff1-b12d-9529b7ef764f None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:31.474481 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f27b68-f2c9-4a29-a570-e53e714eafec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:31.475676 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f27b68-f2c9-4a29-a570-e53e714eafec None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:31.475958 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f27b68-f2c9-4a29-a570-e53e714eafec None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:31.476253 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f27b68-f2c9-4a29-a570-e53e714eafec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:31.478847 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba989f4-b84b-4f3d-8a14-30c54c2100c7 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:31.486973 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f27b68-f2c9-4a29-a570-e53e714eafec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:31.488453 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f27b68-f2c9-4a29-a570-e53e714eafec None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:31.503552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-00f27b68-f2c9-4a29-a570-e53e714eafec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:31.503552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:31.503552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:31.503552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:31.503552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:31.503552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:31.503552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:31.503552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:31.503552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:31.503552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:31.503552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:31.503552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:31.503552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:31.503552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:31.503552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:31.504478 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:31.504478 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:31.504478 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:31.504478 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:31.504478 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:31.504478 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:31.504478 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:31.504478 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:31.504478 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:31.504478 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:31.504478 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:31.504478 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:31.504478 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:31.504478 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:31.504478 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:31.504478 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:31.504478 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:31.505360 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:31.505360 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:31.505360 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:31.505360 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:31.505360 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 276/566] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:31.505821 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2a195-e3b1-4cb2-8f06-a10c988d0a68 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:31.506127 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2a195-e3b1-4cb2-8f06-a10c988d0a68 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:31.506401 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2a195-e3b1-4cb2-8f06-a10c988d0a68 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:31.511390 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 277/567] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:31.511390 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b2a195-e3b1-4cb2-8f06-a10c988d0a68 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:31.515397 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962ae254-077b-42f7-8518-711c23c17162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:31.516055 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ae254-077b-42f7-8518-711c23c17162 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:31.516333 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ae254-077b-42f7-8518-711c23c17162 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:31.516605 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ae254-077b-42f7-8518-711c23c17162 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:31.526846 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962ae254-077b-42f7-8518-711c23c17162 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:38:31.532272 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-962ae254-077b-42f7-8518-711c23c17162 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 12:38:31.532272 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:31.532272 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:31.532272 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:31.532272 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:31.532272 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:31.532272 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:31.532272 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:31.532272 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:31.532272 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:31.532272 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:31.532272 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:31.532272 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:38:31.532272 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:38:31.532272 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:38:31.532272 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:38:31.533430 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:38:31.533430 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:38:31.533430 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:38:31.533430 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:38:31.533430 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:31.533430 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:31.533430 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:31.533430 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:31.533430 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:31.533430 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:31.533430 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:31.533430 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:31.533430 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:31.533430 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:31.533430 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:31.533430 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:31.534266 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:31.534266 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:31.534266 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:31.534266 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:31.534266 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:38:31.534266 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:38:31.534266 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:38:31.534266 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:38:31.534266 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:38:31.534266 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:38:31.534266 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 12:38:31.534266 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:31.534266 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 291/568] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 26 12:38:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:31.534266 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962ae254-077b-42f7-8518-711c23c17162 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:31.569281 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:31.616368 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b2a195-e3b1-4cb2-8f06-a10c988d0a68 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:31.617895 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:31.636446 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962ae254-077b-42f7-8518-711c23c17162 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:31.651123 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c82f4-deb9-4e0b-a6a3-5c25f81b4d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:31.651767 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c82f4-deb9-4e0b-a6a3-5c25f81b4d6b None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:31.652044 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c82f4-deb9-4e0b-a6a3-5c25f81b4d6b None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:31.652314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c82f4-deb9-4e0b-a6a3-5c25f81b4d6b None admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:31.653001 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950c82f4-deb9-4e0b-a6a3-5c25f81b4d6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:31.653301 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950c82f4-deb9-4e0b-a6a3-5c25f81b4d6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:31.654667 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950c82f4-deb9-4e0b-a6a3-5c25f81b4d6b None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:31.666360 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 278/569] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:31.667182 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c82f4-deb9-4e0b-a6a3-5c25f81b4d6b None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:31.670701 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09fc016-bbaf-4f0c-b579-f8bd9a89da2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:31.671386 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fc016-bbaf-4f0c-b579-f8bd9a89da2d None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:31.671696 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fc016-bbaf-4f0c-b579-f8bd9a89da2d None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:31.671984 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fc016-bbaf-4f0c-b579-f8bd9a89da2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:31.683797 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09fc016-bbaf-4f0c-b579-f8bd9a89da2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:31.685260 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09fc016-bbaf-4f0c-b579-f8bd9a89da2d None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:31.699790 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-e09fc016-bbaf-4f0c-b579-f8bd9a89da2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:31.699790 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:31.699790 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:31.699790 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:31.699790 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:31.699790 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:31.699790 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:31.699790 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:31.699790 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:31.699790 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:31.699790 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:31.699790 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:31.699790 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:31.699790 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:31.699790 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:31.700739 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:31.700739 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:31.700739 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:31.700739 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:31.700739 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:31.700739 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:31.700739 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:31.700739 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:31.700739 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:31.700739 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:31.700739 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:31.700739 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:31.700739 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:31.700739 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:31.700739 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:31.700739 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:31.700739 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:31.701622 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:31.701622 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:31.701622 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:31.701622 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:31.705508 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 292/570] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:31.706934 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d4a5d-9b3b-4360-9e7c-7208a25eb78e None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:31.707365 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d4a5d-9b3b-4360-9e7c-7208a25eb78e None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:31.707661 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d4a5d-9b3b-4360-9e7c-7208a25eb78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:31.726246 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-b69d4a5d-9b3b-4360-9e7c-7208a25eb78e None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:31.727578 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-b69d4a5d-9b3b-4360-9e7c-7208a25eb78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:31.768885 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c82f4-deb9-4e0b-a6a3-5c25f81b4d6b None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:31.796908 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 293/571] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:38:31.797763 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d4a5d-9b3b-4360-9e7c-7208a25eb78e None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:31.807789 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c708ba-3b97-45ca-8a3b-8c2bbd8c5fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:31.808479 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c708ba-3b97-45ca-8a3b-8c2bbd8c5fb6 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:31.809623 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c708ba-3b97-45ca-8a3b-8c2bbd8c5fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:31.809623 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c708ba-3b97-45ca-8a3b-8c2bbd8c5fb6 None admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:31.809999 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c708ba-3b97-45ca-8a3b-8c2bbd8c5fb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:31.810309 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c708ba-3b97-45ca-8a3b-8c2bbd8c5fb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:31.811713 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c708ba-3b97-45ca-8a3b-8c2bbd8c5fb6 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:31.823470 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 279/572] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:31.824168 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c708ba-3b97-45ca-8a3b-8c2bbd8c5fb6 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:31.828159 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:31.849361 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:31.916680 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d4a5d-9b3b-4360-9e7c-7208a25eb78e None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:31.929891 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c708ba-3b97-45ca-8a3b-8c2bbd8c5fb6 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:31.950237 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5559f3-7b46-4d2b-b857-7c218e9de9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:31.950846 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5559f3-7b46-4d2b-b857-7c218e9de9d9 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:31.951124 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5559f3-7b46-4d2b-b857-7c218e9de9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:31.951393 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5559f3-7b46-4d2b-b857-7c218e9de9d9 None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:31.952138 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5559f3-7b46-4d2b-b857-7c218e9de9d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:31.952443 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5559f3-7b46-4d2b-b857-7c218e9de9d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:31.953804 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5559f3-7b46-4d2b-b857-7c218e9de9d9 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:31.962449 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 294/573] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Jul 26 12:38:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:31.963120 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5559f3-7b46-4d2b-b857-7c218e9de9d9 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:31.963314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3296a7ad-f5c7-4a76-b461-efaefdbd2e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:31.963928 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3296a7ad-f5c7-4a76-b461-efaefdbd2e11 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:31.964202 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3296a7ad-f5c7-4a76-b461-efaefdbd2e11 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:31.964470 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3296a7ad-f5c7-4a76-b461-efaefdbd2e11 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:31.972848 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3296a7ad-f5c7-4a76-b461-efaefdbd2e11 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:31.974405 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3296a7ad-f5c7-4a76-b461-efaefdbd2e11 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:31.986398 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-3296a7ad-f5c7-4a76-b461-efaefdbd2e11 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:31.986398 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:31.986398 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:31.986398 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:31.986398 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:31.986398 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:31.986398 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:31.986398 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:31.986398 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:31.986398 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:31.986398 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:31.986398 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:31.986398 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 12:38:31.986398 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 12:38:31.986398 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 12:38:31.987422 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 12:38:31.987422 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:31.987422 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:31.987422 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:31.987422 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:31.987422 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:31.987422 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:31.987422 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:31.987422 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:31.987422 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:31.987422 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:31.987422 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:31.987422 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:31.987422 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:31.987422 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:31.987422 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:31.987422 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:31.993461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 12:38:31.993461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 12:38:31.993461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 12:38:31.993461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 12:38:31.993461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 12:38:31.993461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 12:38:31.993461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:31.993461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:31.993461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 280/574] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:38:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:31.993461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3296a7ad-f5c7-4a76-b461-efaefdbd2e11 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:32.049023 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:32.069836 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5559f3-7b46-4d2b-b857-7c218e9de9d9 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:32.070024 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:32.092581 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3296a7ad-f5c7-4a76-b461-efaefdbd2e11 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:32.104880 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec413e00-1291-4746-be81-db22380dae25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:32.105661 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec413e00-1291-4746-be81-db22380dae25 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:32.105948 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec413e00-1291-4746-be81-db22380dae25 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:32.106223 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec413e00-1291-4746-be81-db22380dae25 None admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:32.106920 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec413e00-1291-4746-be81-db22380dae25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:32.107227 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec413e00-1291-4746-be81-db22380dae25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:32.108653 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec413e00-1291-4746-be81-db22380dae25 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:32.121656 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 295/575] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:32.122265 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec413e00-1291-4746-be81-db22380dae25 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:32.132944 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef396cbe-71c3-44d0-8db5-87e6e4a821cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:32.133597 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef396cbe-71c3-44d0-8db5-87e6e4a821cc None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:32.133882 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef396cbe-71c3-44d0-8db5-87e6e4a821cc None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:32.134157 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef396cbe-71c3-44d0-8db5-87e6e4a821cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:32.144624 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef396cbe-71c3-44d0-8db5-87e6e4a821cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:38:32.150583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-ef396cbe-71c3-44d0-8db5-87e6e4a821cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 12:38:32.150583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:32.150583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:32.150583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:32.150583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:32.150583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:32.150583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:32.150583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:32.150583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:32.150583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:32.150583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:32.150583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:32.150583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:38:32.150583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:38:32.150583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:38:32.150583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:38:32.151629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:38:32.151629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:38:32.151629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:38:32.151629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:38:32.151629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:32.151629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:32.151629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:32.151629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:32.151629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:32.151629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:32.151629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:32.151629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:32.151629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:32.151629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:32.151629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:32.151629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:32.152709 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:32.152709 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:32.152709 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:32.152709 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:32.152709 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:38:32.152709 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:38:32.152709 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:38:32.152709 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:38:32.152709 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:38:32.152709 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:38:32.152709 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 12:38:32.152709 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:32.152709 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 281/576] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 26 12:38:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:32.153398 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef396cbe-71c3-44d0-8db5-87e6e4a821cc None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:32.240888 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec413e00-1291-4746-be81-db22380dae25 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:32.254007 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef396cbe-71c3-44d0-8db5-87e6e4a821cc None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:32.276321 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc74cdee-5355-4478-ab37-79337ad10ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:32.276961 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74cdee-5355-4478-ab37-79337ad10ade None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:32.277242 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74cdee-5355-4478-ab37-79337ad10ade None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:32.277524 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74cdee-5355-4478-ab37-79337ad10ade None admin] PATH_INFO: `/v3/projects/service` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:32.284211 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc74cdee-5355-4478-ab37-79337ad10ade None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:32.285650 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc74cdee-5355-4478-ab37-79337ad10ade None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:32.287925 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad84bc-eeff-456f-958b-dbf4c52ebe3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:32.288379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:32.288572 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad84bc-eeff-456f-958b-dbf4c52ebe3b None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:32.288972 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad84bc-eeff-456f-958b-dbf4c52ebe3b None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:32.289119 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad84bc-eeff-456f-958b-dbf4c52ebe3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:32.298811 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-fc74cdee-5355-4478-ab37-79337ad10ade None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:32.298811 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:32.298811 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:32.298811 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:32.298811 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:32.298811 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:32.298811 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:32.298811 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:32.298811 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:32.298811 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:32.298811 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:32.298811 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:32.298811 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 12:38:32.298811 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 12:38:32.298811 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 12:38:32.299774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 12:38:32.299774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:32.299774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:32.299774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:32.299774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:32.299774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:32.299774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:32.299774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:32.299774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:32.299774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:32.299774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:32.299774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:32.299774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:32.299774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:32.299774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:32.299774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:32.299774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:32.300670 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 12:38:32.300670 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 12:38:32.300670 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 12:38:32.300670 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 12:38:32.300670 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 12:38:32.300670 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 12:38:32.300670 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:32.300670 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:32.301797 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ad84bc-eeff-456f-958b-dbf4c52ebe3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:32.301873 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 296/577] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:38:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:32.301873 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc74cdee-5355-4478-ab37-79337ad10ade None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:32.302010 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ad84bc-eeff-456f-958b-dbf4c52ebe3b None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:32.303384 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:32.318353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-22ad84bc-eeff-456f-958b-dbf4c52ebe3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:32.318353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:32.318353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:32.318353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:32.318353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:32.318353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:32.318353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:32.318353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:32.318353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:32.318353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:32.318353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:32.318353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:32.318353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:32.318353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:32.318353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:32.319966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:32.319966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:32.319966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:32.319966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:32.319966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:32.319966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:32.319966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:32.319966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:32.319966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:32.319966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:32.319966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:32.319966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:32.319966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:32.319966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:32.319966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:32.319966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:32.319966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:32.320841 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:32.320841 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:32.320841 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:32.320841 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:32.320841 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 282/578] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:32.320841 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad84bc-eeff-456f-958b-dbf4c52ebe3b None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:32.407383 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc74cdee-5355-4478-ab37-79337ad10ade None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:32.419039 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad84bc-eeff-456f-958b-dbf4c52ebe3b None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:32.440950 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429befca-4555-4571-8c5b-1e11bf06388e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:32.441568 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429befca-4555-4571-8c5b-1e11bf06388e None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:32.441843 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429befca-4555-4571-8c5b-1e11bf06388e None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:32.442112 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429befca-4555-4571-8c5b-1e11bf06388e None admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:32.442845 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429befca-4555-4571-8c5b-1e11bf06388e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:32.443146 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429befca-4555-4571-8c5b-1e11bf06388e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:32.445175 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429befca-4555-4571-8c5b-1e11bf06388e None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:32.455812 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4768dee-363c-4bb3-87b3-fe3e035bf823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:32.456459 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4768dee-363c-4bb3-87b3-fe3e035bf823 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:32.456744 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4768dee-363c-4bb3-87b3-fe3e035bf823 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:32.457020 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4768dee-363c-4bb3-87b3-fe3e035bf823 None admin] PATH_INFO: `/v3/users/glance` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:32.457481 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 297/579] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 26 12:38:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:32.458275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429befca-4555-4571-8c5b-1e11bf06388e None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:32.465625 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4768dee-363c-4bb3-87b3-fe3e035bf823 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:32.467064 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4768dee-363c-4bb3-87b3-fe3e035bf823 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:32.478483 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-c4768dee-363c-4bb3-87b3-fe3e035bf823 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 26 12:38:32.478483 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:32.478483 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:32.478483 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:32.478483 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:32.478483 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:32.478483 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:32.478483 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:32.478483 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:32.478483 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:32.478483 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:32.478483 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:32.478483 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 12:38:32.478483 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 12:38:32.478483 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 12:38:32.478483 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 12:38:32.479592 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:32.479592 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:32.479592 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 12:38:32.479592 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:32.479592 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 12:38:32.479592 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:32.479592 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:32.479592 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:32.479592 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:32.479592 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:32.479592 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:32.479592 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:32.479592 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:32.479592 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:32.479592 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:32.479592 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:32.479592 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:32.479592 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:32.485417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:32.485417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:32.485417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 12:38:32.485417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 12:38:32.485417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 12:38:32.485417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 12:38:32.485417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 12:38:32.485417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 12:38:32.485417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 26 12:38:32.485417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:32.485417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 283/580] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:38:32 2022] GET /identity/v3/users/glance => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:32.485417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4768dee-363c-4bb3-87b3-fe3e035bf823 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:32.518198 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:32.535468 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:32.560452 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429befca-4555-4571-8c5b-1e11bf06388e None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:32.581146 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4768dee-363c-4bb3-87b3-fe3e035bf823 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:32.594710 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea70ada-c611-4b31-9438-a9f7a4c0f3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:32.595404 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea70ada-c611-4b31-9438-a9f7a4c0f3de None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:32.595726 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea70ada-c611-4b31-9438-a9f7a4c0f3de None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:32.596007 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea70ada-c611-4b31-9438-a9f7a4c0f3de None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:32.596722 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea70ada-c611-4b31-9438-a9f7a4c0f3de None admin] RBAC: Adding query filter params (name=service) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:32.597029 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea70ada-c611-4b31-9438-a9f7a4c0f3de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:32.598395 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea70ada-c611-4b31-9438-a9f7a4c0f3de None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:32.607005 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 298/581] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Jul 26 12:38:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:32.608047 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea70ada-c611-4b31-9438-a9f7a4c0f3de None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:32.615024 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac801a55-68ba-41b2-8587-09219878ee86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:32.615683 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac801a55-68ba-41b2-8587-09219878ee86 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:32.615963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac801a55-68ba-41b2-8587-09219878ee86 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:32.616234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac801a55-68ba-41b2-8587-09219878ee86 None admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:32.616972 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac801a55-68ba-41b2-8587-09219878ee86 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:32.617274 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac801a55-68ba-41b2-8587-09219878ee86 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:32.618700 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac801a55-68ba-41b2-8587-09219878ee86 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:32.634541 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 284/582] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 26 12:38:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:32.634541 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac801a55-68ba-41b2-8587-09219878ee86 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:32.713909 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea70ada-c611-4b31-9438-a9f7a4c0f3de None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:32.732877 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac801a55-68ba-41b2-8587-09219878ee86 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:32.735785 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:32.747661 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:32.749046 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdddb23-e444-4970-92c6-1fba465cbb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:32.749706 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdddb23-e444-4970-92c6-1fba465cbb41 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:32.749989 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdddb23-e444-4970-92c6-1fba465cbb41 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:32.750266 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdddb23-e444-4970-92c6-1fba465cbb41 None admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:32.750960 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdddb23-e444-4970-92c6-1fba465cbb41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:32.751264 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdddb23-e444-4970-92c6-1fba465cbb41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:32.752675 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdddb23-e444-4970-92c6-1fba465cbb41 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:32.765273 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 299/583] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:32.765949 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdddb23-e444-4970-92c6-1fba465cbb41 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:32.768577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92029275-3a1c-4a88-87c2-c28f2e840235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:32.769167 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92029275-3a1c-4a88-87c2-c28f2e840235 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:32.769451 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92029275-3a1c-4a88-87c2-c28f2e840235 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:32.769730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92029275-3a1c-4a88-87c2-c28f2e840235 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:32.770516 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92029275-3a1c-4a88-87c2-c28f2e840235 None admin] RBAC: Adding query filter params (role.id=9c68629183974be9bfb1ed95f69912ee, scope.project.id=644278e7e8934b37a1e7ab20d35764cd, user.id=cd2483f74d7d48b1a7b8ffe32f63abde) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:32.770824 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92029275-3a1c-4a88-87c2-c28f2e840235 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:32.772341 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92029275-3a1c-4a88-87c2-c28f2e840235 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:32.777830 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 285/584] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:38:32 2022] GET /identity/v3/role_assignments?user.id=cd2483f74d7d48b1a7b8ffe32f63abde&scope.project.id=644278e7e8934b37a1e7ab20d35764cd&role.id=9c68629183974be9bfb1ed95f69912ee => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:32.778602 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92029275-3a1c-4a88-87c2-c28f2e840235 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:32.893532 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdddb23-e444-4970-92c6-1fba465cbb41 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:32.900973 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92029275-3a1c-4a88-87c2-c28f2e840235 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:32.935558 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db717f9-d250-4538-bdc7-6d1eda0c5b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:32.936297 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db717f9-d250-4538-bdc7-6d1eda0c5b72 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:32.936572 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db717f9-d250-4538-bdc7-6d1eda0c5b72 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:32.936852 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db717f9-d250-4538-bdc7-6d1eda0c5b72 None admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:32.937649 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db717f9-d250-4538-bdc7-6d1eda0c5b72 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:32.937966 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db717f9-d250-4538-bdc7-6d1eda0c5b72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:32.939407 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db717f9-d250-4538-bdc7-6d1eda0c5b72 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:32.942326 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087287da-284f-45fb-a8b3-a40533bf70a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:32.942972 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087287da-284f-45fb-a8b3-a40533bf70a0 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:32.943252 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087287da-284f-45fb-a8b3-a40533bf70a0 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:32.944069 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087287da-284f-45fb-a8b3-a40533bf70a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:32.955714 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087287da-284f-45fb-a8b3-a40533bf70a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:32.957162 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087287da-284f-45fb-a8b3-a40533bf70a0 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:32.963173 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 300/585] 10.222.0.20 () {56 vars in 1137 bytes} [Tue Jul 26 12:38:32 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:32.963173 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db717f9-d250-4538-bdc7-6d1eda0c5b72 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:32.978239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-087287da-284f-45fb-a8b3-a40533bf70a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:32.978239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:32.978239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:32.978239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:32.978239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:32.978239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:32.978239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:32.978239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:32.978239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:32.978239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:32.978239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:32.978239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:32.978239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:32.978239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:32.978239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:32.979155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:32.979155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:32.979155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:32.979155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:32.979155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:32.979155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:32.979155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:32.979155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:32.979155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:32.979155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:32.979155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:32.979155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:32.979155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:32.979155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:32.979155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:32.979155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:32.979155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:32.980069 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:32.980069 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:32.980069 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:32.980069 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:32.980069 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 286/586] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:32.980345 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087287da-284f-45fb-a8b3-a40533bf70a0 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:33.012977 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:33.013219 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:33.077874 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db717f9-d250-4538-bdc7-6d1eda0c5b72 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:33.090686 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087287da-284f-45fb-a8b3-a40533bf70a0 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:33.122552 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815864b0-f4e5-485d-a772-ec431b33ca0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:33.123228 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815864b0-f4e5-485d-a772-ec431b33ca0a None admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:33.123512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815864b0-f4e5-485d-a772-ec431b33ca0a None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:33.123838 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815864b0-f4e5-485d-a772-ec431b33ca0a None admin] PATH_INFO: `/v3/projects/644278e7e8934b37a1e7ab20d35764cd/users/0f5d6f3c12244c0faf26048513dc49e7/roles/9c68629183974be9bfb1ed95f69912ee` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:33.127886 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69595045-9991-41b1-9511-21bb8aef0aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:33.128521 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69595045-9991-41b1-9511-21bb8aef0aec None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:33.128804 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69595045-9991-41b1-9511-21bb8aef0aec None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:33.129079 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69595045-9991-41b1-9511-21bb8aef0aec None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:33.138163 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69595045-9991-41b1-9511-21bb8aef0aec None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:33.139807 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69595045-9991-41b1-9511-21bb8aef0aec None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:33.140524 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815864b0-f4e5-485d-a772-ec431b33ca0a None admin] RBAC: Authorizing `identity:create_grant(project_id=644278e7e8934b37a1e7ab20d35764cd, user_id=0f5d6f3c12244c0faf26048513dc49e7, role_id=9c68629183974be9bfb1ed95f69912ee)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:33.141981 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815864b0-f4e5-485d-a772-ec431b33ca0a None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:33.153130 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-69595045-9991-41b1-9511-21bb8aef0aec None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 12:38:33.153130 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:33.153130 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:33.153130 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:33.153130 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:33.153130 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:33.153130 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:33.153130 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:33.153130 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:33.153130 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:33.153130 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:33.153130 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:33.153130 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 12:38:33.153130 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 12:38:33.153130 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 12:38:33.153130 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 12:38:33.154178 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:33.154178 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:33.154178 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 12:38:33.154178 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:33.154178 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 12:38:33.154178 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:33.154178 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:33.154178 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:33.154178 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:33.154178 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:33.154178 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:33.154178 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:33.154178 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:33.154178 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:33.154178 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:33.154178 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:33.154178 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:33.154178 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:33.155126 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:33.155126 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:33.155126 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 12:38:33.155126 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 12:38:33.155126 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 12:38:33.155126 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 12:38:33.155126 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 12:38:33.155126 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 12:38:33.155126 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 12:38:33.155126 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:33.164100 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 301/587] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 26 12:38:32 2022] PUT /identity/v3/projects/644278e7e8934b37a1e7ab20d35764cd/users/0f5d6f3c12244c0faf26048513dc49e7/roles/9c68629183974be9bfb1ed95f69912ee => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:38:33.164100 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815864b0-f4e5-485d-a772-ec431b33ca0a None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:33.164425 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 287/588] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:38:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:33.164425 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69595045-9991-41b1-9511-21bb8aef0aec None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:33.292093 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815864b0-f4e5-485d-a772-ec431b33ca0a None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:33.301461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:33.308107 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:33.317380 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69595045-9991-41b1-9511-21bb8aef0aec None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:33.352024 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb886b8-e091-4abc-af3c-6752fba1dd25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:33.352760 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb886b8-e091-4abc-af3c-6752fba1dd25 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:33.353113 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb886b8-e091-4abc-af3c-6752fba1dd25 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:33.353458 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb886b8-e091-4abc-af3c-6752fba1dd25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:33.364906 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30cdfd2-df58-45d5-91ee-ce2911044527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:33.366485 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30cdfd2-df58-45d5-91ee-ce2911044527 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:33.366592 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb886b8-e091-4abc-af3c-6752fba1dd25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:33.366765 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30cdfd2-df58-45d5-91ee-ce2911044527 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:33.367034 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30cdfd2-df58-45d5-91ee-ce2911044527 None admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:33.367754 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30cdfd2-df58-45d5-91ee-ce2911044527 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:33.368057 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30cdfd2-df58-45d5-91ee-ce2911044527 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:33.368165 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb886b8-e091-4abc-af3c-6752fba1dd25 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:33.369463 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30cdfd2-df58-45d5-91ee-ce2911044527 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:33.385021 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-dcb886b8-e091-4abc-af3c-6752fba1dd25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:33.385021 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:33.385021 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:33.385021 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:33.385021 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:33.385021 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:33.385021 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:33.385021 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:33.385021 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:33.385021 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:33.385021 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:33.385021 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:33.385021 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:33.385021 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:33.385021 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:33.386039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:33.386039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:33.386039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:33.386039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:33.386039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:33.386039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:33.386039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:33.386039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:33.386039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:33.386039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:33.386039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:33.386039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:33.386039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:33.386039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:33.386039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:33.386039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:33.386039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:33.386902 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:33.386902 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:33.386902 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:33.386902 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:33.386902 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 302/589] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:33.387174 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb886b8-e091-4abc-af3c-6752fba1dd25 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:33.390965 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 288/590] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:33.393331 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30cdfd2-df58-45d5-91ee-ce2911044527 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:33.510673 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb886b8-e091-4abc-af3c-6752fba1dd25 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:33.518589 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30cdfd2-df58-45d5-91ee-ce2911044527 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:33.548362 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba7d96-8f19-47de-84ff-6db6befb28f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:33.549067 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba7d96-8f19-47de-84ff-6db6befb28f3 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:33.549353 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba7d96-8f19-47de-84ff-6db6befb28f3 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:33.549627 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba7d96-8f19-47de-84ff-6db6befb28f3 None admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:33.550429 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba7d96-8f19-47de-84ff-6db6befb28f3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:33.550737 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba7d96-8f19-47de-84ff-6db6befb28f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:33.552257 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba7d96-8f19-47de-84ff-6db6befb28f3 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:33.563716 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f9dbc-455e-413b-841b-f3112e93a51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:33.564859 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f9dbc-455e-413b-841b-f3112e93a51b None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:33.566251 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f9dbc-455e-413b-841b-f3112e93a51b None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:33.566706 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f9dbc-455e-413b-841b-f3112e93a51b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:33.572257 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:33.576016 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6f9dbc-455e-413b-841b-f3112e93a51b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:33.577520 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6f9dbc-455e-413b-841b-f3112e93a51b None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:33.585590 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 303/591] 10.222.0.20 () {56 vars in 1137 bytes} [Tue Jul 26 12:38:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:33.585590 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba7d96-8f19-47de-84ff-6db6befb28f3 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:33.598281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-8a6f9dbc-455e-413b-841b-f3112e93a51b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 12:38:33.598281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:33.598281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:33.598281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:33.598281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:33.598281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:33.598281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:33.598281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:33.598281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:33.598281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:33.598281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:33.598281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:33.598281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 12:38:33.598281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 12:38:33.598281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 12:38:33.598281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 12:38:33.599316 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:33.599316 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:33.599316 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 12:38:33.599316 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:33.599316 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 12:38:33.599316 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:33.599316 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:33.599316 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:33.599316 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:33.599316 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:33.599316 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:33.599316 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:33.599316 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:33.599316 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:33.599316 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:33.599316 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:33.599316 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:33.599316 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:33.600352 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:33.600352 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:33.600352 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 12:38:33.600352 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 12:38:33.600352 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 12:38:33.600352 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 12:38:33.600352 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 12:38:33.600352 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 12:38:33.600352 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 12:38:33.600352 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:33.604098 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 289/592] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 26 12:38:33 2022] GET /identity/v3/users/neutron => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:38:33.605085 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f9dbc-455e-413b-841b-f3112e93a51b None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:33.618674 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:33.696595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba7d96-8f19-47de-84ff-6db6befb28f3 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:33.719665 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f9dbc-455e-413b-841b-f3112e93a51b None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:33.732547 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2744c8b-736f-40df-b3a8-cd921755c118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:33.733489 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2744c8b-736f-40df-b3a8-cd921755c118 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:33.733964 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2744c8b-736f-40df-b3a8-cd921755c118 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:33.734435 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2744c8b-736f-40df-b3a8-cd921755c118 None admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:33.735316 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2744c8b-736f-40df-b3a8-cd921755c118 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:33.735835 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2744c8b-736f-40df-b3a8-cd921755c118 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:33.737404 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2744c8b-736f-40df-b3a8-cd921755c118 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:33.754746 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 304/593] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:33.756052 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2744c8b-736f-40df-b3a8-cd921755c118 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:33.759830 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3241138-2606-4422-914d-92cb9ca79e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:33.760482 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3241138-2606-4422-914d-92cb9ca79e92 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:33.760784 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3241138-2606-4422-914d-92cb9ca79e92 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:33.761814 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3241138-2606-4422-914d-92cb9ca79e92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:33.773279 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3241138-2606-4422-914d-92cb9ca79e92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:33.774791 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3241138-2606-4422-914d-92cb9ca79e92 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:33.791184 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-d3241138-2606-4422-914d-92cb9ca79e92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:33.791184 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:33.791184 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:33.791184 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:33.791184 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:33.791184 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:33.791184 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:33.791184 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:33.791184 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:33.791184 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:33.791184 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:33.791184 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:33.791184 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:33.791184 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:33.791184 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:33.792317 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:33.792317 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:33.792317 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:33.792317 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:33.792317 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:33.792317 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:33.792317 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:33.792317 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:33.792317 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:33.792317 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:33.792317 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:33.792317 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:33.792317 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:33.792317 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:33.792317 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:33.792317 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:33.792317 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:33.793172 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:33.793172 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:33.793172 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:33.793172 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:33.797112 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 290/594] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:33.798514 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3241138-2606-4422-914d-92cb9ca79e92 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:33.832150 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:33.864719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:33.880413 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2744c8b-736f-40df-b3a8-cd921755c118 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:33.918468 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3241138-2606-4422-914d-92cb9ca79e92 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:33.925351 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a0e79-217d-4bec-8f26-5e947fe31178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:33.926053 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a0e79-217d-4bec-8f26-5e947fe31178 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:33.926406 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a0e79-217d-4bec-8f26-5e947fe31178 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:33.926755 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a0e79-217d-4bec-8f26-5e947fe31178 None admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:33.927618 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87a0e79-217d-4bec-8f26-5e947fe31178 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:33.928978 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87a0e79-217d-4bec-8f26-5e947fe31178 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:33.930110 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87a0e79-217d-4bec-8f26-5e947fe31178 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:33.953896 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 305/595] 10.222.0.20 () {56 vars in 1139 bytes} [Tue Jul 26 12:38:33 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:33.954906 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a0e79-217d-4bec-8f26-5e947fe31178 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:33.958758 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df99359-8eed-4236-8db4-c13aa5eea1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:33.959689 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df99359-8eed-4236-8db4-c13aa5eea1e2 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:33.960173 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df99359-8eed-4236-8db4-c13aa5eea1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:33.960621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df99359-8eed-4236-8db4-c13aa5eea1e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:33.969833 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df99359-8eed-4236-8db4-c13aa5eea1e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:33.971628 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df99359-8eed-4236-8db4-c13aa5eea1e2 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:33.985712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-6df99359-8eed-4236-8db4-c13aa5eea1e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:33.985712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:33.985712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:33.985712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:33.985712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:33.985712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:33.985712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:33.985712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:33.985712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:33.985712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:33.985712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:33.985712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:33.985712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 12:38:33.985712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 12:38:33.985712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 12:38:33.986638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 12:38:33.986638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:33.986638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:33.986638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:33.986638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:33.986638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:33.986638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:33.986638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:33.986638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:33.986638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:33.986638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:33.986638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:33.986638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:33.986638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:33.986638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:33.986638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:33.986638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:33.987495 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 12:38:33.987495 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 12:38:33.987495 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 12:38:33.987495 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 12:38:33.987495 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 12:38:33.987495 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 12:38:33.987495 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:33.987495 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:33.995104 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 291/596] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:38:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:33.996414 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df99359-8eed-4236-8db4-c13aa5eea1e2 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:34.067430 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a0e79-217d-4bec-8f26-5e947fe31178 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:34.085975 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:34.105103 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16923e3c-8ecb-4a90-b086-567a5f6e22c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:34.105804 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16923e3c-8ecb-4a90-b086-567a5f6e22c2 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:34.106156 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16923e3c-8ecb-4a90-b086-567a5f6e22c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:34.106628 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16923e3c-8ecb-4a90-b086-567a5f6e22c2 None admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:34.107251 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16923e3c-8ecb-4a90-b086-567a5f6e22c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:34.107623 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16923e3c-8ecb-4a90-b086-567a5f6e22c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:34.109105 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16923e3c-8ecb-4a90-b086-567a5f6e22c2 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:34.110001 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df99359-8eed-4236-8db4-c13aa5eea1e2 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:34.121225 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 306/597] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:34.126419 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16923e3c-8ecb-4a90-b086-567a5f6e22c2 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:34.135105 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:34.149302 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8470a3c-a061-48cb-a7dd-2833f1d1c156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:34.150218 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8470a3c-a061-48cb-a7dd-2833f1d1c156 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:34.150772 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8470a3c-a061-48cb-a7dd-2833f1d1c156 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:34.151217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8470a3c-a061-48cb-a7dd-2833f1d1c156 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:34.164416 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8470a3c-a061-48cb-a7dd-2833f1d1c156 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:34.166165 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8470a3c-a061-48cb-a7dd-2833f1d1c156 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:34.183234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-f8470a3c-a061-48cb-a7dd-2833f1d1c156 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:34.183234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:34.183234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:34.183234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:34.183234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:34.183234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:34.183234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:34.183234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:34.183234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:34.183234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:34.183234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:34.183234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:34.183234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:34.183234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:34.183234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:34.184308 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:34.184308 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:34.184308 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:34.184308 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:34.184308 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:34.184308 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:34.184308 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:34.184308 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:34.184308 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:34.184308 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:34.184308 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:34.184308 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:34.184308 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:34.184308 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:34.184308 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:34.184308 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:34.184308 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:34.185156 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:34.185156 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:34.185156 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:34.185156 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:34.187402 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 292/598] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:34.188379 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8470a3c-a061-48cb-a7dd-2833f1d1c156 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:34.237884 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16923e3c-8ecb-4a90-b086-567a5f6e22c2 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:34.274440 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d0d543-1119-4a71-872c-b5fbf0215068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:34.275423 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0d543-1119-4a71-872c-b5fbf0215068 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:34.275423 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0d543-1119-4a71-872c-b5fbf0215068 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:34.275799 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0d543-1119-4a71-872c-b5fbf0215068 None admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:34.276783 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d0d543-1119-4a71-872c-b5fbf0215068 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:34.277166 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d0d543-1119-4a71-872c-b5fbf0215068 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:34.278664 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d0d543-1119-4a71-872c-b5fbf0215068 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:34.296656 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 307/599] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 26 12:38:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:34.297800 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d0d543-1119-4a71-872c-b5fbf0215068 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:34.302531 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8470a3c-a061-48cb-a7dd-2833f1d1c156 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:34.340285 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:34.342427 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e0dfd-4338-46d6-a079-1677187df527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:34.344292 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e0dfd-4338-46d6-a079-1677187df527 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:34.344810 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e0dfd-4338-46d6-a079-1677187df527 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:34.345381 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e0dfd-4338-46d6-a079-1677187df527 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:34.353594 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1e0dfd-4338-46d6-a079-1677187df527 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:34.355334 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1e0dfd-4338-46d6-a079-1677187df527 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:34.369311 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-bd1e0dfd-4338-46d6-a079-1677187df527 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:34.369311 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:34.369311 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:34.369311 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:34.369311 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:34.369311 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:34.369311 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:34.369311 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:34.369311 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:34.369311 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:34.369311 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:34.369311 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:34.369311 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 12:38:34.369311 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 12:38:34.369311 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 12:38:34.370274 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 12:38:34.370274 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:34.370274 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:34.370274 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:34.370274 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:34.370274 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:34.370274 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:34.370274 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:34.370274 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:34.370274 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:34.370274 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:34.370274 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:34.370274 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:34.370274 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:34.370274 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:34.370274 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:34.370274 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:34.371165 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 12:38:34.371165 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 12:38:34.371165 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 12:38:34.371165 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 12:38:34.371165 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 12:38:34.371165 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 12:38:34.371165 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:34.371165 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:34.373198 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 293/600] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:38:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:34.374394 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e0dfd-4338-46d6-a079-1677187df527 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:34.409590 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d0d543-1119-4a71-872c-b5fbf0215068 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:34.415712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:34.447107 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13fe34f-c145-4d72-b542-076ed5261b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:34.447853 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13fe34f-c145-4d72-b542-076ed5261b2b None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:34.448207 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13fe34f-c145-4d72-b542-076ed5261b2b None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:34.448643 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13fe34f-c145-4d72-b542-076ed5261b2b None admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:34.449319 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13fe34f-c145-4d72-b542-076ed5261b2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:34.449955 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13fe34f-c145-4d72-b542-076ed5261b2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:34.451295 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13fe34f-c145-4d72-b542-076ed5261b2b None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:34.467940 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 308/601] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:34.468961 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13fe34f-c145-4d72-b542-076ed5261b2b None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:34.492270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e0dfd-4338-46d6-a079-1677187df527 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:34.531854 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653831fb-5156-419a-83cf-ab163df37a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:34.532876 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653831fb-5156-419a-83cf-ab163df37a1f None admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:34.533430 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653831fb-5156-419a-83cf-ab163df37a1f None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:34.533902 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653831fb-5156-419a-83cf-ab163df37a1f None admin] PATH_INFO: `/v3/projects/644278e7e8934b37a1e7ab20d35764cd/users/d2b5853838dc40db89c46038c0606107/roles/9c68629183974be9bfb1ed95f69912ee` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:34.551877 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653831fb-5156-419a-83cf-ab163df37a1f None admin] RBAC: Authorizing `identity:create_grant(project_id=644278e7e8934b37a1e7ab20d35764cd, user_id=d2b5853838dc40db89c46038c0606107, role_id=9c68629183974be9bfb1ed95f69912ee)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:34.554010 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653831fb-5156-419a-83cf-ab163df37a1f None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:34.570248 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 294/602] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 26 12:38:34 2022] PUT /identity/v3/projects/644278e7e8934b37a1e7ab20d35764cd/users/d2b5853838dc40db89c46038c0606107/roles/9c68629183974be9bfb1ed95f69912ee => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:38:34.571567 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653831fb-5156-419a-83cf-ab163df37a1f None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:34.593541 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:34.604080 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13fe34f-c145-4d72-b542-076ed5261b2b None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:34.669419 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b0ec4-2a99-47e0-a56c-be3937af174a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:34.670076 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b0ec4-2a99-47e0-a56c-be3937af174a None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:34.670356 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b0ec4-2a99-47e0-a56c-be3937af174a None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:34.670624 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b0ec4-2a99-47e0-a56c-be3937af174a None admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:34.671399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1b0ec4-2a99-47e0-a56c-be3937af174a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:34.671712 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1b0ec4-2a99-47e0-a56c-be3937af174a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:34.673166 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1b0ec4-2a99-47e0-a56c-be3937af174a None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:34.675922 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:34.698063 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653831fb-5156-419a-83cf-ab163df37a1f None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:34.699630 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 309/603] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 26 12:38:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:34.704722 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b0ec4-2a99-47e0-a56c-be3937af174a None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:34.741401 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89443e2a-4fae-4d61-838a-7b5c76136857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:34.742485 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89443e2a-4fae-4d61-838a-7b5c76136857 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:34.743092 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89443e2a-4fae-4d61-838a-7b5c76136857 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:34.743680 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89443e2a-4fae-4d61-838a-7b5c76136857 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:34.751910 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89443e2a-4fae-4d61-838a-7b5c76136857 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:34.753805 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89443e2a-4fae-4d61-838a-7b5c76136857 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:34.768100 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-89443e2a-4fae-4d61-838a-7b5c76136857 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:34.768100 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:34.768100 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:34.768100 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:34.768100 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:34.768100 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:34.768100 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:34.768100 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:34.768100 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:34.768100 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:34.768100 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:34.768100 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:34.768100 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 12:38:34.768100 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 12:38:34.768100 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 12:38:34.769043 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 12:38:34.769043 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:34.769043 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:34.769043 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:34.769043 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:34.769043 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:34.769043 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:34.769043 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:34.769043 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:34.769043 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:34.769043 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:34.769043 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:34.769043 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:34.769043 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:34.769043 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:34.769043 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:34.769043 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:34.769905 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 12:38:34.769905 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 12:38:34.769905 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 12:38:34.769905 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 12:38:34.769905 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 12:38:34.769905 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 12:38:34.769905 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:34.769905 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:34.773107 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 295/604] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:38:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:34.789876 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89443e2a-4fae-4d61-838a-7b5c76136857 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:34.835064 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b0ec4-2a99-47e0-a56c-be3937af174a None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:34.876319 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520d7606-0b4e-44ba-9a58-4ad422b1a4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:34.877171 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d7606-0b4e-44ba-9a58-4ad422b1a4dd None admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:34.877394 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d7606-0b4e-44ba-9a58-4ad422b1a4dd None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:34.877670 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d7606-0b4e-44ba-9a58-4ad422b1a4dd None admin] PATH_INFO: `/v3/projects/644278e7e8934b37a1e7ab20d35764cd/users/9f8ca5e019dd4b9090cee2b2c1d8e15d/roles/9c68629183974be9bfb1ed95f69912ee` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:34.896123 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520d7606-0b4e-44ba-9a58-4ad422b1a4dd None admin] RBAC: Authorizing `identity:create_grant(project_id=644278e7e8934b37a1e7ab20d35764cd, user_id=9f8ca5e019dd4b9090cee2b2c1d8e15d, role_id=9c68629183974be9bfb1ed95f69912ee)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:34.898152 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520d7606-0b4e-44ba-9a58-4ad422b1a4dd None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:34.914037 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 310/605] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 26 12:38:34 2022] PUT /identity/v3/projects/644278e7e8934b37a1e7ab20d35764cd/users/9f8ca5e019dd4b9090cee2b2c1d8e15d/roles/9c68629183974be9bfb1ed95f69912ee => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:38:34.915545 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d401b8b-beab-4385-80b4-b080f7fe4126 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:34.915864 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d401b8b-beab-4385-80b4-b080f7fe4126 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:34.916246 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d401b8b-beab-4385-80b4-b080f7fe4126 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:34.917799 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 311/606] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:34.926831 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e7aeb-d27f-48ff-b280-f57d6f8daf86 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:34.927207 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e7aeb-d27f-48ff-b280-f57d6f8daf86 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:34.927601 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e7aeb-d27f-48ff-b280-f57d6f8daf86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:34.946494 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89443e2a-4fae-4d61-838a-7b5c76136857 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:34.956743 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-814e7aeb-d27f-48ff-b280-f57d6f8daf86 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:34.958249 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-814e7aeb-d27f-48ff-b280-f57d6f8daf86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:34.961356 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:35.001798 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea4c34-c7c8-4ae9-8eee-ffba500c54d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:35.002638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea4c34-c7c8-4ae9-8eee-ffba500c54d9 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:35.003074 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea4c34-c7c8-4ae9-8eee-ffba500c54d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:35.003452 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea4c34-c7c8-4ae9-8eee-ffba500c54d9 None admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:35.004381 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ea4c34-c7c8-4ae9-8eee-ffba500c54d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:35.004814 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ea4c34-c7c8-4ae9-8eee-ffba500c54d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:35.006964 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ea4c34-c7c8-4ae9-8eee-ffba500c54d9 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:35.027230 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 296/607] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 26 12:38:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:38:35.029370 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea4c34-c7c8-4ae9-8eee-ffba500c54d9 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:35.034417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:35.063205 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 312/608] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 12:38:35.091315 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814e7aeb-d27f-48ff-b280-f57d6f8daf86 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:35.180824 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea4c34-c7c8-4ae9-8eee-ffba500c54d9 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:35.226269 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ff7d39-aa5c-4091-99cc-b15d6c21fa3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:35.226618 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814e7aeb-d27f-48ff-b280-f57d6f8daf86 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:35.227485 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff7d39-aa5c-4091-99cc-b15d6c21fa3e None admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:35.228254 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff7d39-aa5c-4091-99cc-b15d6c21fa3e None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:35.228729 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff7d39-aa5c-4091-99cc-b15d6c21fa3e None admin] PATH_INFO: `/v3/projects/644278e7e8934b37a1e7ab20d35764cd/users/6830350c6eba4610bbe9ab1dc31f5939/roles/9c68629183974be9bfb1ed95f69912ee` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:35.250292 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ff7d39-aa5c-4091-99cc-b15d6c21fa3e None admin] RBAC: Authorizing `identity:create_grant(project_id=644278e7e8934b37a1e7ab20d35764cd, user_id=6830350c6eba4610bbe9ab1dc31f5939, role_id=9c68629183974be9bfb1ed95f69912ee)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:35.252229 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ff7d39-aa5c-4091-99cc-b15d6c21fa3e None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:35.274097 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 297/609] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 26 12:38:35 2022] PUT /identity/v3/projects/644278e7e8934b37a1e7ab20d35764cd/users/6830350c6eba4610bbe9ab1dc31f5939/roles/9c68629183974be9bfb1ed95f69912ee => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:38:35.284225 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef216ee-1178-424b-920d-3c8a7c597593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:35.284914 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef216ee-1178-424b-920d-3c8a7c597593 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:35.285293 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef216ee-1178-424b-920d-3c8a7c597593 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:35.285575 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef216ee-1178-424b-920d-3c8a7c597593 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:35.303674 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:35.308822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef216ee-1178-424b-920d-3c8a7c597593 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:38:35.317171 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-8ef216ee-1178-424b-920d-3c8a7c597593 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 12:38:35.317171 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:35.317171 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:35.317171 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:35.317171 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:35.317171 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:35.317171 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:35.317171 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:35.317171 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:35.317171 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:35.317171 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:35.317171 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:35.317171 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:38:35.317171 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:38:35.317171 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:38:35.317171 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:38:35.318254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:38:35.318254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:38:35.318254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:38:35.318254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:38:35.318254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:35.318254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:35.318254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:35.318254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:35.318254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:35.318254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:35.318254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:35.318254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:35.318254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:35.318254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:35.318254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:35.318254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:35.319124 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:35.319124 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:35.319124 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:35.319124 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:35.319124 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:38:35.319124 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:38:35.319124 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:38:35.319124 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:38:35.319124 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:38:35.319124 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:38:35.319124 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 12:38:35.319124 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:35.319124 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 313/610] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 26 12:38:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:35.326364 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ff7d39-aa5c-4091-99cc-b15d6c21fa3e None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:35.425670 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:35.477033 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ff7d39-aa5c-4091-99cc-b15d6c21fa3e None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:35.519205 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ed4efc-d27d-4716-8177-1d38db4fe4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:35.520386 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed4efc-d27d-4716-8177-1d38db4fe4b1 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:35.520517 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1200b7ec-0850-41cf-ae19-647068c25a74 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:35.520746 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1200b7ec-0850-41cf-ae19-647068c25a74 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:35.521027 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1200b7ec-0850-41cf-ae19-647068c25a74 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:35.521417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed4efc-d27d-4716-8177-1d38db4fe4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:35.521924 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed4efc-d27d-4716-8177-1d38db4fe4b1 None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:35.522855 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ed4efc-d27d-4716-8177-1d38db4fe4b1 None admin] RBAC: Adding query filter params (name=service) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:35.523362 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ed4efc-d27d-4716-8177-1d38db4fe4b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:35.523814 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 314/611] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 12:38:35.525223 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ed4efc-d27d-4716-8177-1d38db4fe4b1 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:35.528911 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a1ea8-762c-4904-b055-c9ff8435357b None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:35.528911 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a1ea8-762c-4904-b055-c9ff8435357b None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:35.528911 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a1ea8-762c-4904-b055-c9ff8435357b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:35.541523 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 298/612] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Jul 26 12:38:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:35.547310 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ed4efc-d27d-4716-8177-1d38db4fe4b1 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:35.549749 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-d48a1ea8-762c-4904-b055-c9ff8435357b None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:35.553691 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-d48a1ea8-762c-4904-b055-c9ff8435357b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:35.647731 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 315/613] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 12:38:35.670588 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ed4efc-d27d-4716-8177-1d38db4fe4b1 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:35.687250 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a1ea8-762c-4904-b055-c9ff8435357b None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:35.697638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:35.756546 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f06f72e-3f28-472a-adc5-194a7ef84499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:35.757215 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f06f72e-3f28-472a-adc5-194a7ef84499 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:35.757564 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f06f72e-3f28-472a-adc5-194a7ef84499 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:35.758053 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f06f72e-3f28-472a-adc5-194a7ef84499 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:35.781709 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f06f72e-3f28-472a-adc5-194a7ef84499 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:35.783286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f06f72e-3f28-472a-adc5-194a7ef84499 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:35.808003 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a1ea8-762c-4904-b055-c9ff8435357b None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:35.813137 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-1f06f72e-3f28-472a-adc5-194a7ef84499 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:35.813137 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:35.813137 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:35.813137 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:35.813137 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:35.813137 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:35.813137 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:35.813137 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:35.813137 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:35.813137 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:35.813137 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:35.813137 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:35.813137 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:35.813137 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:35.813137 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:35.814023 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:35.814023 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:35.814023 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:35.814023 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:35.814023 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:35.814023 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:35.814023 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:35.814023 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:35.814023 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:35.814023 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:35.814023 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:35.814023 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:35.814023 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:35.814023 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:35.814023 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:35.814023 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:35.814023 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:35.814902 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:35.814902 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:35.814902 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:35.814902 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:35.823998 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 299/614] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:38:35.824936 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f06f72e-3f28-472a-adc5-194a7ef84499 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:35.836896 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:35.845952 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c15ba7-07d2-4282-900a-49ae7e44aa2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:35.846601 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c15ba7-07d2-4282-900a-49ae7e44aa2e None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:35.846889 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c15ba7-07d2-4282-900a-49ae7e44aa2e None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:35.847171 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c15ba7-07d2-4282-900a-49ae7e44aa2e None admin] PATH_INFO: `/v3/roles/service` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:35.868947 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c15ba7-07d2-4282-900a-49ae7e44aa2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:38:35.874861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-a9c15ba7-07d2-4282-900a-49ae7e44aa2e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 12:38:35.874861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:35.874861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:35.874861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:35.874861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:35.874861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:35.874861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:35.874861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:35.874861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:35.874861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:35.874861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:35.874861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:35.874861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:38:35.874861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:38:35.874861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:38:35.874861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:38:35.876102 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:38:35.876102 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:38:35.876102 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:38:35.876102 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:38:35.876102 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:35.876102 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:35.876102 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:35.876102 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:35.876102 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:35.876102 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:35.876102 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:35.876102 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:35.876102 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:35.876102 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:35.876102 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:35.876102 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:35.877145 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:35.877145 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:35.877145 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:35.877145 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:35.877145 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:38:35.877145 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:38:35.877145 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:38:35.877145 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:38:35.877145 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:38:35.877145 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:38:35.877145 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 12:38:35.877145 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:35.882601 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 316/615] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 26 12:38:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:38:35.885770 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c15ba7-07d2-4282-900a-49ae7e44aa2e None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:36.041445 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c15ba7-07d2-4282-900a-49ae7e44aa2e None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:36.053710 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f06f72e-3f28-472a-adc5-194a7ef84499 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:36.079150 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a5492f-164c-4981-8e3a-7169e395c5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:36.080096 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5492f-164c-4981-8e3a-7169e395c5a8 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:36.080518 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5492f-164c-4981-8e3a-7169e395c5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:36.080934 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5492f-164c-4981-8e3a-7169e395c5a8 None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:36.081847 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a5492f-164c-4981-8e3a-7169e395c5a8 None admin] RBAC: Adding query filter params (name=service) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:36.082397 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a5492f-164c-4981-8e3a-7169e395c5a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:36.083989 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a5492f-164c-4981-8e3a-7169e395c5a8 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:36.092998 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 317/616] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Jul 26 12:38:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:36.099094 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a5492f-164c-4981-8e3a-7169e395c5a8 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:36.130034 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee16d93-5477-4354-8a51-4aa8d63d87f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:36.131506 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee16d93-5477-4354-8a51-4aa8d63d87f3 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:36.131976 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee16d93-5477-4354-8a51-4aa8d63d87f3 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:36.132408 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee16d93-5477-4354-8a51-4aa8d63d87f3 None admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:36.134740 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee16d93-5477-4354-8a51-4aa8d63d87f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:36.135242 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee16d93-5477-4354-8a51-4aa8d63d87f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:36.136970 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee16d93-5477-4354-8a51-4aa8d63d87f3 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:36.166414 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 300/617] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:36.171700 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee16d93-5477-4354-8a51-4aa8d63d87f3 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:36.198989 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:36.226509 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:36.304641 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a5492f-164c-4981-8e3a-7169e395c5a8 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:36.324282 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee16d93-5477-4354-8a51-4aa8d63d87f3 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:36.360674 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133be2ba-c9ae-4e78-91d3-595ba89a135e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:36.361314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133be2ba-c9ae-4e78-91d3-595ba89a135e None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:36.361586 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133be2ba-c9ae-4e78-91d3-595ba89a135e None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:36.361974 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133be2ba-c9ae-4e78-91d3-595ba89a135e None admin] PATH_INFO: `/v3/users/swift` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:36.369946 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133be2ba-c9ae-4e78-91d3-595ba89a135e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:36.371395 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133be2ba-c9ae-4e78-91d3-595ba89a135e None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:36.380331 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba9c195-87d2-4e33-8eb9-57c78373e220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:36.381025 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9c195-87d2-4e33-8eb9-57c78373e220 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:36.381305 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9c195-87d2-4e33-8eb9-57c78373e220 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:36.381578 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9c195-87d2-4e33-8eb9-57c78373e220 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:36.384188 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-133be2ba-c9ae-4e78-91d3-595ba89a135e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 12:38:36.384188 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:36.384188 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:36.384188 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:36.384188 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:36.384188 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:36.384188 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:36.384188 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:36.384188 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:36.384188 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:36.384188 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:36.384188 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:36.384188 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 12:38:36.384188 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 12:38:36.384188 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 12:38:36.384188 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 12:38:36.385171 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:36.385171 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:36.385171 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 12:38:36.385171 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:36.385171 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 12:38:36.385171 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:36.385171 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:36.385171 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:36.385171 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:36.385171 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:36.385171 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:36.385171 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:36.385171 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:36.385171 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:36.385171 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:36.385171 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:36.385171 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:36.385171 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:36.386148 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:36.386148 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:36.386148 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 12:38:36.386148 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 12:38:36.386148 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 12:38:36.386148 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 12:38:36.386148 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 12:38:36.386148 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 12:38:36.386148 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 12:38:36.386148 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:36.386148 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 301/618] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 26 12:38:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:38:36.397204 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133be2ba-c9ae-4e78-91d3-595ba89a135e None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:36.408351 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba9c195-87d2-4e33-8eb9-57c78373e220 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:36.409953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba9c195-87d2-4e33-8eb9-57c78373e220 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:36.428517 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-fba9c195-87d2-4e33-8eb9-57c78373e220 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:36.428517 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:36.428517 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:36.428517 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:36.428517 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:36.428517 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:36.428517 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:36.428517 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:36.428517 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:36.428517 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:36.428517 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:36.428517 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:36.428517 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:36.428517 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:36.428517 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:36.430499 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:36.430499 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:36.430499 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:36.430499 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:36.430499 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:36.430499 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:36.430499 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:36.430499 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:36.430499 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:36.430499 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:36.430499 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:36.430499 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:36.430499 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:36.430499 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:36.430499 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:36.430499 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:36.430499 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:36.432008 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:36.432008 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:36.432008 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:36.432008 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:36.432008 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 318/619] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:36.445170 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba9c195-87d2-4e33-8eb9-57c78373e220 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:36.504326 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:36.510903 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133be2ba-c9ae-4e78-91d3-595ba89a135e None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:36.549712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825bb77b-4d76-468a-a07e-ca83a01d7f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:36.550341 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825bb77b-4d76-468a-a07e-ca83a01d7f0c None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:36.550625 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825bb77b-4d76-468a-a07e-ca83a01d7f0c None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:36.550903 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825bb77b-4d76-468a-a07e-ca83a01d7f0c None admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:36.551759 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825bb77b-4d76-468a-a07e-ca83a01d7f0c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:36.552058 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825bb77b-4d76-468a-a07e-ca83a01d7f0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:36.553550 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825bb77b-4d76-468a-a07e-ca83a01d7f0c None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:36.573425 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 302/620] 10.222.0.20 () {56 vars in 1135 bytes} [Tue Jul 26 12:38:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:36.579437 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825bb77b-4d76-468a-a07e-ca83a01d7f0c None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:36.643239 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba9c195-87d2-4e33-8eb9-57c78373e220 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:36.646265 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:36.714046 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3794b44a-c020-491b-a29d-a2c4205307a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:36.714741 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3794b44a-c020-491b-a29d-a2c4205307a9 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:36.715024 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3794b44a-c020-491b-a29d-a2c4205307a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:36.715303 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3794b44a-c020-491b-a29d-a2c4205307a9 None admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:36.718818 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3794b44a-c020-491b-a29d-a2c4205307a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:36.719137 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3794b44a-c020-491b-a29d-a2c4205307a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:36.720555 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3794b44a-c020-491b-a29d-a2c4205307a9 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:36.746177 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 319/621] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:36.751127 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3794b44a-c020-491b-a29d-a2c4205307a9 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:36.765186 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825bb77b-4d76-468a-a07e-ca83a01d7f0c None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:36.801511 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce6f17-60f9-42ba-98eb-6f973cc133b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:36.802247 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce6f17-60f9-42ba-98eb-6f973cc133b0 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:36.802564 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce6f17-60f9-42ba-98eb-6f973cc133b0 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:36.802887 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce6f17-60f9-42ba-98eb-6f973cc133b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:36.814810 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce6f17-60f9-42ba-98eb-6f973cc133b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:36.816314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce6f17-60f9-42ba-98eb-6f973cc133b0 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:36.832730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-99ce6f17-60f9-42ba-98eb-6f973cc133b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:36.832730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:36.832730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:36.832730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:36.832730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:36.832730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:36.832730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:36.832730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:36.832730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:36.832730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:36.832730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:36.832730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:36.832730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:36.832730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:36.832730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:36.833637 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:36.833637 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:36.833637 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:36.833637 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:36.833637 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:36.833637 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:36.833637 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:36.833637 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:36.833637 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:36.833637 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:36.833637 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:36.833637 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:36.833637 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:36.833637 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:36.833637 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:36.833637 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:36.833637 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:36.834621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:36.834621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:36.834621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:36.834621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:36.834621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 303/622] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:38:36.835920 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:36.841665 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce6f17-60f9-42ba-98eb-6f973cc133b0 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:36.957057 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce6f17-60f9-42ba-98eb-6f973cc133b0 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:36.992368 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc4e9b1-d36f-4474-b6fa-dcfc3564e3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:36.992994 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4e9b1-d36f-4474-b6fa-dcfc3564e3ed None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:36.993282 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4e9b1-d36f-4474-b6fa-dcfc3564e3ed None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:36.993565 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4e9b1-d36f-4474-b6fa-dcfc3564e3ed None admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:36.994265 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc4e9b1-d36f-4474-b6fa-dcfc3564e3ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:36.994574 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc4e9b1-d36f-4474-b6fa-dcfc3564e3ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:36.996191 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc4e9b1-d36f-4474-b6fa-dcfc3564e3ed None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:37.013291 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 304/623] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:38:37.020163 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc4e9b1-d36f-4474-b6fa-dcfc3564e3ed None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:37.036108 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3794b44a-c020-491b-a29d-a2c4205307a9 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:37.101346 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:37.134787 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5beaab8-e6f8-43cc-bdc2-155dd20436ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:37.135443 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5beaab8-e6f8-43cc-bdc2-155dd20436ea None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:37.135751 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5beaab8-e6f8-43cc-bdc2-155dd20436ea None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:37.136029 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5beaab8-e6f8-43cc-bdc2-155dd20436ea None admin] PATH_INFO: `/v3/users/nova` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:37.147277 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5beaab8-e6f8-43cc-bdc2-155dd20436ea None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:37.149153 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5beaab8-e6f8-43cc-bdc2-155dd20436ea None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:37.164079 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-d5beaab8-e6f8-43cc-bdc2-155dd20436ea None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 12:38:37.164079 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:37.164079 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:37.164079 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:37.164079 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:37.164079 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:37.164079 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:37.164079 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:37.164079 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:37.164079 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:37.164079 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:37.164079 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:37.164079 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 12:38:37.164079 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 12:38:37.164079 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 12:38:37.164079 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 12:38:37.165083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:37.165083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:37.165083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 12:38:37.165083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:37.165083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 12:38:37.165083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:37.165083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:37.165083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:37.165083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:37.165083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:37.165083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:37.165083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:37.165083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:37.165083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:37.165083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:37.165083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:37.165083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:37.165083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:37.166026 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:37.166026 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:37.166026 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 12:38:37.166026 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 12:38:37.166026 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 12:38:37.166026 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 12:38:37.166026 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 12:38:37.166026 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 12:38:37.166026 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 12:38:37.166026 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:37.166026 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 320/624] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Jul 26 12:38:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 415 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:38:37.171555 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558bd498-96df-4f10-b3f3-68bd8d83b63a None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:37.171555 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558bd498-96df-4f10-b3f3-68bd8d83b63a None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:37.171555 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558bd498-96df-4f10-b3f3-68bd8d83b63a None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:37.171879 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc4e9b1-d36f-4474-b6fa-dcfc3564e3ed None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:37.173901 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 321/625] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:37.174709 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558bd498-96df-4f10-b3f3-68bd8d83b63a None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:37.214158 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1988b0b-42f2-496e-abbb-4bc7d06efe09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:37.214834 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1988b0b-42f2-496e-abbb-4bc7d06efe09 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:37.215187 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1988b0b-42f2-496e-abbb-4bc7d06efe09 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:37.215467 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1988b0b-42f2-496e-abbb-4bc7d06efe09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:37.220065 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:37.222969 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1988b0b-42f2-496e-abbb-4bc7d06efe09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:37.224484 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1988b0b-42f2-496e-abbb-4bc7d06efe09 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:37.236921 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-e1988b0b-42f2-496e-abbb-4bc7d06efe09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:37.236921 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:37.236921 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:37.236921 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:37.236921 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:37.236921 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:37.236921 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:37.236921 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:37.236921 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:37.236921 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:37.236921 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:37.236921 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:37.236921 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 12:38:37.236921 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 12:38:37.236921 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 12:38:37.237902 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 12:38:37.237902 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:37.237902 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:37.237902 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:37.237902 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:37.237902 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:37.237902 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:37.237902 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:37.237902 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:37.237902 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:37.237902 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:37.237902 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:37.237902 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:37.237902 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:37.237902 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:37.237902 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:37.237902 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:37.238784 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 12:38:37.238784 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 12:38:37.238784 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 12:38:37.238784 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 12:38:37.238784 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 12:38:37.238784 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 12:38:37.238784 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:37.238784 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:37.243691 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 305/626] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:38:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:37.244990 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168293b9-5534-4264-8749-4775423b469c None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:37.245279 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168293b9-5534-4264-8749-4775423b469c None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:37.245559 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168293b9-5534-4264-8749-4775423b469c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:37.264028 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-168293b9-5534-4264-8749-4775423b469c None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:37.265243 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-168293b9-5534-4264-8749-4775423b469c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:37.297115 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558bd498-96df-4f10-b3f3-68bd8d83b63a None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:37.335850 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c385999-41df-4a4f-92ab-83de730b270d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:37.336481 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 306/627] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:38:37.337663 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c385999-41df-4a4f-92ab-83de730b270d None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:37.338201 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c385999-41df-4a4f-92ab-83de730b270d None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:37.338691 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c385999-41df-4a4f-92ab-83de730b270d None admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:37.339733 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c385999-41df-4a4f-92ab-83de730b270d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:37.340246 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c385999-41df-4a4f-92ab-83de730b270d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:37.341984 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c385999-41df-4a4f-92ab-83de730b270d None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:37.344229 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168293b9-5534-4264-8749-4775423b469c None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:37.362249 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 322/628] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Jul 26 12:38:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:37.372738 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c385999-41df-4a4f-92ab-83de730b270d None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:37.404534 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:37.457051 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168293b9-5534-4264-8749-4775423b469c None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:37.476000 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:37.490717 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c385999-41df-4a4f-92ab-83de730b270d None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:37.493817 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b1f065-3ca8-4e31-aa4d-cde9af801056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:37.494198 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b1f065-3ca8-4e31-aa4d-cde9af801056 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:37.494492 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b1f065-3ca8-4e31-aa4d-cde9af801056 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:37.494759 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b1f065-3ca8-4e31-aa4d-cde9af801056 None admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:37.495512 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b1f065-3ca8-4e31-aa4d-cde9af801056 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:37.495850 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b1f065-3ca8-4e31-aa4d-cde9af801056 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:37.497296 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b1f065-3ca8-4e31-aa4d-cde9af801056 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:37.514443 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 307/629] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 26 12:38:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:37.514443 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b1f065-3ca8-4e31-aa4d-cde9af801056 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:37.536198 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad5779-c452-4133-abb3-343287f39ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:37.536898 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad5779-c452-4133-abb3-343287f39ba3 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:37.537242 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad5779-c452-4133-abb3-343287f39ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:37.537518 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad5779-c452-4133-abb3-343287f39ba3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:37.608524 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ad5779-c452-4133-abb3-343287f39ba3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:37.610007 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ad5779-c452-4133-abb3-343287f39ba3 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:37.624056 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b1f065-3ca8-4e31-aa4d-cde9af801056 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:37.641776 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:37.676520 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222bac15-ce6f-4477-8ad6-0c79eaee8860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:37.677275 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222bac15-ce6f-4477-8ad6-0c79eaee8860 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:37.677585 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222bac15-ce6f-4477-8ad6-0c79eaee8860 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:37.677962 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222bac15-ce6f-4477-8ad6-0c79eaee8860 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:37.689366 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222bac15-ce6f-4477-8ad6-0c79eaee8860 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:38:37.695338 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-222bac15-ce6f-4477-8ad6-0c79eaee8860 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 12:38:37.695338 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:37.695338 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:37.695338 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:37.695338 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:37.695338 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:37.695338 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:37.695338 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:37.695338 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:37.695338 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:37.695338 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:37.695338 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:37.695338 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:38:37.695338 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:38:37.695338 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:38:37.695338 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:38:37.696367 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:38:37.696367 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:38:37.696367 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:38:37.696367 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:38:37.696367 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:37.696367 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:37.696367 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:37.696367 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:37.696367 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:37.696367 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:37.696367 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:37.696367 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:37.696367 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:37.696367 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:37.696367 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:37.696367 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:37.702177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:37.702177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:37.702177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:37.702177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:37.702177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:38:37.702177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:38:37.702177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:38:37.702177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:38:37.702177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:38:37.702177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:38:37.702177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 12:38:37.702177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:37.702177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 308/630] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 26 12:38:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:37.702177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624bdf6c-c565-4059-9b7f-e2e1e0541a54 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:37.703013 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624bdf6c-c565-4059-9b7f-e2e1e0541a54 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:37.703013 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624bdf6c-c565-4059-9b7f-e2e1e0541a54 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:37.703156 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:37.704451 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 309/631] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:37.705372 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624bdf6c-c565-4059-9b7f-e2e1e0541a54 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:37.722479 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 323/632] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:38:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 12:38:37.726142 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad5779-c452-4133-abb3-343287f39ba3 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:37.809198 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624bdf6c-c565-4059-9b7f-e2e1e0541a54 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:37.825334 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad5779-c452-4133-abb3-343287f39ba3 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:37.843051 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a0915a-ed1c-42d7-b897-421f7b245e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:37.843751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0915a-ed1c-42d7-b897-421f7b245e18 None admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:37.844031 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0915a-ed1c-42d7-b897-421f7b245e18 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:37.844309 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0915a-ed1c-42d7-b897-421f7b245e18 None admin] PATH_INFO: `/v3/projects/644278e7e8934b37a1e7ab20d35764cd/users/cd2483f74d7d48b1a7b8ffe32f63abde/roles/9c68629183974be9bfb1ed95f69912ee` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:37.859979 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbd668d-5210-44f8-92b5-c3cd392b587a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:37.860169 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a0915a-ed1c-42d7-b897-421f7b245e18 None admin] RBAC: Authorizing `identity:create_grant(project_id=644278e7e8934b37a1e7ab20d35764cd, user_id=cd2483f74d7d48b1a7b8ffe32f63abde, role_id=9c68629183974be9bfb1ed95f69912ee)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:37.860587 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbd668d-5210-44f8-92b5-c3cd392b587a None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:37.860866 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbd668d-5210-44f8-92b5-c3cd392b587a None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:37.861146 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbd668d-5210-44f8-92b5-c3cd392b587a None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:37.861606 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a0915a-ed1c-42d7-b897-421f7b245e18 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:37.861856 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbd668d-5210-44f8-92b5-c3cd392b587a None admin] RBAC: Adding query filter params (name=service) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:37.862156 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbd668d-5210-44f8-92b5-c3cd392b587a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:37.863505 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbd668d-5210-44f8-92b5-c3cd392b587a None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:37.874544 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 324/633] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Jul 26 12:38:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:37.875915 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a51661-6b63-46c7-9a50-b1a27de6622f None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:37.876042 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a51661-6b63-46c7-9a50-b1a27de6622f None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:37.876149 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 310/634] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 26 12:38:37 2022] PUT /identity/v3/projects/644278e7e8934b37a1e7ab20d35764cd/users/cd2483f74d7d48b1a7b8ffe32f63abde/roles/9c68629183974be9bfb1ed95f69912ee => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:38:37.876372 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a51661-6b63-46c7-9a50-b1a27de6622f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:37.877348 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827c79f-c7c1-4d80-a9d7-8b894d730150 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:37.877637 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827c79f-c7c1-4d80-a9d7-8b894d730150 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:37.877919 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827c79f-c7c1-4d80-a9d7-8b894d730150 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:37.879428 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 311/635] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:37.880213 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827c79f-c7c1-4d80-a9d7-8b894d730150 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:37.901364 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-b0a51661-6b63-46c7-9a50-b1a27de6622f None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:37.902922 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-b0a51661-6b63-46c7-9a50-b1a27de6622f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:37.911771 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:37.945040 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:37.983604 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 325/636] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:38:37.985174 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c369620-9b27-4825-a271-32b345b595eb None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:37.985633 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c369620-9b27-4825-a271-32b345b595eb None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:37.986053 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c369620-9b27-4825-a271-32b345b595eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:38.013849 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-0c369620-9b27-4825-a271-32b345b595eb None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:38.014418 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-0c369620-9b27-4825-a271-32b345b595eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:38.029124 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827c79f-c7c1-4d80-a9d7-8b894d730150 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:38.068765 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0604557-e6f1-405e-a10f-bf3e908743a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:38.069759 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0604557-e6f1-405e-a10f-bf3e908743a5 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:38.070293 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0604557-e6f1-405e-a10f-bf3e908743a5 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:38.070769 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0604557-e6f1-405e-a10f-bf3e908743a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:38.081893 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0604557-e6f1-405e-a10f-bf3e908743a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:38.084605 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0604557-e6f1-405e-a10f-bf3e908743a5 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:38.092032 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 326/637] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:38:38.092883 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c369620-9b27-4825-a271-32b345b595eb None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:38.106232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-b0604557-e6f1-405e-a10f-bf3e908743a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:38.106232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:38.106232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:38.106232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:38.106232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:38.106232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:38.106232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:38.106232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:38.106232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:38.106232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:38.106232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:38.106232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:38.106232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:38.106232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:38.106232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:38.107163 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:38.107163 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:38.107163 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:38.107163 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:38.107163 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:38.107163 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:38.107163 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:38.107163 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:38.107163 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:38.107163 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:38.107163 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:38.107163 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:38.107163 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:38.107163 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:38.107163 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:38.107163 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:38.107163 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:38.108094 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:38.108094 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:38.108094 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:38.108094 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:38.108886 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 312/638] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:38.111662 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0604557-e6f1-405e-a10f-bf3e908743a5 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:38.202896 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c369620-9b27-4825-a271-32b345b595eb None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:38.204490 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:38.205224 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:38.241448 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0604557-e6f1-405e-a10f-bf3e908743a5 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:38.242628 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f7f57-6ce3-4116-8a9d-421d7e24ed80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:38.243283 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f7f57-6ce3-4116-8a9d-421d7e24ed80 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:38.243568 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f7f57-6ce3-4116-8a9d-421d7e24ed80 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:38.243856 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f7f57-6ce3-4116-8a9d-421d7e24ed80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:38.254409 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015f7f57-6ce3-4116-8a9d-421d7e24ed80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:38.256547 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015f7f57-6ce3-4116-8a9d-421d7e24ed80 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:38.272152 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-015f7f57-6ce3-4116-8a9d-421d7e24ed80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:38.272152 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:38.272152 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:38.272152 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:38.272152 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:38.272152 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:38.272152 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:38.272152 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:38.272152 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:38.272152 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:38.272152 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:38.272152 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:38.272152 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:38.272152 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:38.272152 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:38.273077 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:38.273077 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:38.273077 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:38.273077 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:38.273077 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:38.273077 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:38.273077 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:38.273077 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:38.273077 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:38.273077 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:38.273077 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:38.273077 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:38.273077 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:38.273077 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:38.273077 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:38.273077 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:38.273077 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:38.273950 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:38.273950 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:38.273950 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:38.273950 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:38.273950 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 327/639] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:38.274238 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f7f57-6ce3-4116-8a9d-421d7e24ed80 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:38.284684 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f04730-4e50-4694-90b3-60d899311cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:38.286062 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f04730-4e50-4694-90b3-60d899311cff None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:38.286955 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f04730-4e50-4694-90b3-60d899311cff None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:38.287961 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f04730-4e50-4694-90b3-60d899311cff None admin] PATH_INFO: `/v3/roles/service` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:38.300733 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f04730-4e50-4694-90b3-60d899311cff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:38:38.306237 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-a5f04730-4e50-4694-90b3-60d899311cff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 12:38:38.306237 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:38.306237 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:38.306237 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:38.306237 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:38.306237 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:38.306237 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:38.306237 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:38.306237 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:38.306237 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:38.306237 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:38.306237 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:38.306237 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:38:38.306237 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:38:38.306237 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:38:38.306237 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:38:38.307239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:38:38.307239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:38:38.307239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:38:38.307239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:38:38.307239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:38.307239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:38.307239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:38.307239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:38.307239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:38.307239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:38.307239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:38.307239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:38.307239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:38.307239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:38.307239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:38.307239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:38.308103 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:38.308103 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:38.308103 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:38.308103 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:38.308103 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:38:38.308103 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:38:38.308103 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:38:38.308103 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:38:38.308103 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:38:38.308103 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:38:38.308103 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 12:38:38.308103 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:38.308103 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 313/640] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 26 12:38:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:38.308811 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f04730-4e50-4694-90b3-60d899311cff None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:38.386978 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f7f57-6ce3-4116-8a9d-421d7e24ed80 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:38.419036 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f04730-4e50-4694-90b3-60d899311cff None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:38.422554 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf998524-e796-4488-911e-3f548b88aa0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:38.423171 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf998524-e796-4488-911e-3f548b88aa0a None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:38.423455 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf998524-e796-4488-911e-3f548b88aa0a None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:38.423760 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf998524-e796-4488-911e-3f548b88aa0a None admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:38.424452 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf998524-e796-4488-911e-3f548b88aa0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:38.424758 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf998524-e796-4488-911e-3f548b88aa0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:38.426134 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf998524-e796-4488-911e-3f548b88aa0a None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:38.438995 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 328/641] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:38.439578 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf998524-e796-4488-911e-3f548b88aa0a None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:38.456276 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:38.456437 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:38.463631 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54afa1f7-e5d2-43ba-ba34-af7c3f6af1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:38.464737 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54afa1f7-e5d2-43ba-ba34-af7c3f6af1b8 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:38.465024 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54afa1f7-e5d2-43ba-ba34-af7c3f6af1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:38.465380 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54afa1f7-e5d2-43ba-ba34-af7c3f6af1b8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:38.476659 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54afa1f7-e5d2-43ba-ba34-af7c3f6af1b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:38:38.482040 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-54afa1f7-e5d2-43ba-ba34-af7c3f6af1b8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 12:38:38.482040 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:38.482040 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:38.482040 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:38.482040 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:38.482040 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:38.482040 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:38.482040 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:38.482040 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:38.482040 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:38.482040 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:38.482040 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:38.482040 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:38:38.482040 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:38:38.482040 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:38:38.482040 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:38:38.483055 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:38:38.483055 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:38:38.483055 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:38:38.483055 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:38:38.483055 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:38.483055 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:38.483055 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:38.483055 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:38.483055 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:38.483055 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:38.483055 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:38.483055 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:38.483055 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:38.483055 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:38.483055 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:38.483055 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:38.483926 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:38.483926 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:38.483926 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:38.483926 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:38.483926 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:38:38.483926 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:38:38.483926 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:38:38.483926 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:38:38.483926 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:38:38.483926 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:38:38.483926 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 12:38:38.483926 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:38.483926 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 314/642] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 26 12:38:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:38.484643 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54afa1f7-e5d2-43ba-ba34-af7c3f6af1b8 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:38.552713 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf998524-e796-4488-911e-3f548b88aa0a None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:38.592462 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fdf6a0-bb82-4565-b7ac-a789ea019b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:38.593078 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fdf6a0-bb82-4565-b7ac-a789ea019b16 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:38.593367 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fdf6a0-bb82-4565-b7ac-a789ea019b16 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:38.593643 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fdf6a0-bb82-4565-b7ac-a789ea019b16 None admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:38.594337 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fdf6a0-bb82-4565-b7ac-a789ea019b16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:38.594645 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fdf6a0-bb82-4565-b7ac-a789ea019b16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:38.596070 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fdf6a0-bb82-4565-b7ac-a789ea019b16 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:38.598860 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54afa1f7-e5d2-43ba-ba34-af7c3f6af1b8 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:38.607900 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 329/643] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:38.608717 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fdf6a0-bb82-4565-b7ac-a789ea019b16 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:38.639767 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b383e-7ec5-49ab-9fca-cd77a3d502cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:38.640723 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b383e-7ec5-49ab-9fca-cd77a3d502cd None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:38.641185 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b383e-7ec5-49ab-9fca-cd77a3d502cd None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:38.641640 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b383e-7ec5-49ab-9fca-cd77a3d502cd None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:38.642535 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16b383e-7ec5-49ab-9fca-cd77a3d502cd None admin] RBAC: Adding query filter params (name=service) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:38.643045 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16b383e-7ec5-49ab-9fca-cd77a3d502cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:38.644436 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16b383e-7ec5-49ab-9fca-cd77a3d502cd None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:38.654342 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 315/644] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Jul 26 12:38:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:38.655172 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b383e-7ec5-49ab-9fca-cd77a3d502cd None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:38.696746 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:38.716691 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:38.728049 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fdf6a0-bb82-4565-b7ac-a789ea019b16 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:38.763615 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a118b-9d4a-455a-8cb5-4cf465767c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:38.764313 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a118b-9d4a-455a-8cb5-4cf465767c85 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:38.764598 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a118b-9d4a-455a-8cb5-4cf465767c85 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:38.764876 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a118b-9d4a-455a-8cb5-4cf465767c85 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:38.772238 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5a118b-9d4a-455a-8cb5-4cf465767c85 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:38.772761 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b383e-7ec5-49ab-9fca-cd77a3d502cd None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:38.773691 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5a118b-9d4a-455a-8cb5-4cf465767c85 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:38.786130 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-1e5a118b-9d4a-455a-8cb5-4cf465767c85 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:38.786130 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:38.786130 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:38.786130 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:38.786130 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:38.786130 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:38.786130 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:38.786130 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:38.786130 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:38.786130 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:38.786130 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:38.786130 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:38.786130 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 12:38:38.786130 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 12:38:38.786130 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 12:38:38.787108 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 12:38:38.787108 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:38.787108 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:38.787108 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:38.787108 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:38.787108 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:38.787108 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:38.787108 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:38.787108 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:38.787108 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:38.787108 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:38.787108 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:38.787108 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:38.787108 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:38.787108 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:38.787108 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:38.787108 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:38.788263 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 12:38:38.788263 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 12:38:38.788263 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 12:38:38.788263 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 12:38:38.788263 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 12:38:38.788263 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 12:38:38.788263 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:38.788263 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:38.788263 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 330/645] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:38:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:38.788263 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a118b-9d4a-455a-8cb5-4cf465767c85 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:38.816125 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e304a41e-4ac5-4f7d-ad90-1bbc17d34f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:38.816748 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e304a41e-4ac5-4f7d-ad90-1bbc17d34f6c None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:38.817030 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e304a41e-4ac5-4f7d-ad90-1bbc17d34f6c None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:38.817387 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e304a41e-4ac5-4f7d-ad90-1bbc17d34f6c None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:38.818022 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e304a41e-4ac5-4f7d-ad90-1bbc17d34f6c None admin] RBAC: Adding query filter params (name=service) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:38.818354 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e304a41e-4ac5-4f7d-ad90-1bbc17d34f6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:38.820028 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e304a41e-4ac5-4f7d-ad90-1bbc17d34f6c None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:38.828194 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 316/646] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Jul 26 12:38:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:38.828540 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e304a41e-4ac5-4f7d-ad90-1bbc17d34f6c None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:38.902032 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a118b-9d4a-455a-8cb5-4cf465767c85 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:38.937785 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deefaf3c-27af-4353-9bea-b973eb327883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:38.938515 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e304a41e-4ac5-4f7d-ad90-1bbc17d34f6c None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:38.939024 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefaf3c-27af-4353-9bea-b973eb327883 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:38.939522 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefaf3c-27af-4353-9bea-b973eb327883 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:38.940036 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefaf3c-27af-4353-9bea-b973eb327883 None admin] PATH_INFO: `/v3/users/glance` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:38.942331 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:38.951161 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deefaf3c-27af-4353-9bea-b973eb327883 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:38.953185 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deefaf3c-27af-4353-9bea-b973eb327883 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:38.956341 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:38.975342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-deefaf3c-27af-4353-9bea-b973eb327883 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 26 12:38:38.975342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:38.975342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:38.975342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:38.975342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:38.975342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:38.975342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:38.975342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:38.975342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:38.975342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:38.975342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:38.975342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:38.975342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 12:38:38.975342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 12:38:38.975342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 12:38:38.975342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 12:38:38.976547 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:38.976547 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:38.976547 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 12:38:38.976547 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:38.976547 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 12:38:38.976547 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:38.976547 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:38.976547 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:38.976547 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:38.976547 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:38.976547 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:38.976547 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:38.976547 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:38.976547 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:38.976547 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:38.976547 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:38.976547 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:38.976547 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:38.977667 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:38.977667 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:38.977667 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 12:38:38.977667 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 12:38:38.977667 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 12:38:38.977667 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 12:38:38.977667 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 12:38:38.977667 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 12:38:38.977667 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 26 12:38:38.977667 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:38.980017 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 331/647] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:38:38 2022] GET /identity/v3/users/glance => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:38.981227 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deefaf3c-27af-4353-9bea-b973eb327883 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:38.981381 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8709639-faa4-4180-83f4-f383e6429656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:38.982061 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8709639-faa4-4180-83f4-f383e6429656 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:38.982393 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8709639-faa4-4180-83f4-f383e6429656 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:38.982617 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8709639-faa4-4180-83f4-f383e6429656 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:38.995205 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8709639-faa4-4180-83f4-f383e6429656 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:38.996721 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8709639-faa4-4180-83f4-f383e6429656 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:39.013619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-b8709639-faa4-4180-83f4-f383e6429656 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:39.013619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:39.013619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:39.013619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:39.013619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:39.013619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:39.013619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:39.013619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:39.013619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:39.013619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:39.013619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:39.013619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:39.013619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:39.013619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:39.013619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:39.014578 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:39.014578 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:39.014578 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:39.014578 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:39.014578 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:39.014578 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:39.014578 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:39.014578 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:39.014578 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:39.014578 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:39.014578 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:39.014578 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:39.014578 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:39.014578 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:39.014578 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:39.014578 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:39.014578 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:39.015467 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:39.015467 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:39.015467 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:39.015467 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:39.015467 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 317/648] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:39.016369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8709639-faa4-4180-83f4-f383e6429656 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:39.097061 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deefaf3c-27af-4353-9bea-b973eb327883 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:39.119393 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8709639-faa4-4180-83f4-f383e6429656 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:39.135936 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83284538-a674-4bc6-9b0c-57e5f5a2987b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:39.136547 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83284538-a674-4bc6-9b0c-57e5f5a2987b None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:39.136828 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83284538-a674-4bc6-9b0c-57e5f5a2987b None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:39.137099 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83284538-a674-4bc6-9b0c-57e5f5a2987b None admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:39.137849 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83284538-a674-4bc6-9b0c-57e5f5a2987b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:39.138155 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83284538-a674-4bc6-9b0c-57e5f5a2987b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:39.139582 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83284538-a674-4bc6-9b0c-57e5f5a2987b None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:39.158476 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 332/649] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 26 12:38:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:39.158476 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83284538-a674-4bc6-9b0c-57e5f5a2987b None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:39.162767 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc94257-bdce-4625-ad60-c8951071540f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:39.163552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc94257-bdce-4625-ad60-c8951071540f None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:39.163931 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc94257-bdce-4625-ad60-c8951071540f None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:39.164280 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc94257-bdce-4625-ad60-c8951071540f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:39.178107 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc94257-bdce-4625-ad60-c8951071540f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:39.179624 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc94257-bdce-4625-ad60-c8951071540f None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:39.195234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-bcc94257-bdce-4625-ad60-c8951071540f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:39.195234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:39.195234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:39.195234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:39.195234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:39.195234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:39.195234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:39.195234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:39.195234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:39.195234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:39.195234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:39.195234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:39.195234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:39.195234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:39.195234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:39.196198 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:39.196198 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:39.196198 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:39.196198 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:39.196198 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:39.196198 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:39.196198 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:39.196198 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:39.196198 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:39.196198 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:39.196198 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:39.196198 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:39.196198 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:39.196198 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:39.196198 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:39.196198 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:39.196198 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:39.197099 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:39.197099 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:39.197099 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:39.197099 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:39.197099 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 318/650] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:39.197383 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc94257-bdce-4625-ad60-c8951071540f None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:39.219611 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:39.233682 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:39.280209 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83284538-a674-4bc6-9b0c-57e5f5a2987b None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:39.311816 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc94257-bdce-4625-ad60-c8951071540f None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:39.317161 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494c2c4-82b5-4c19-a876-cb30addc2562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:39.317778 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494c2c4-82b5-4c19-a876-cb30addc2562 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:39.318055 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494c2c4-82b5-4c19-a876-cb30addc2562 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:39.318375 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494c2c4-82b5-4c19-a876-cb30addc2562 None admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:39.319179 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494c2c4-82b5-4c19-a876-cb30addc2562 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:39.319483 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494c2c4-82b5-4c19-a876-cb30addc2562 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:39.321415 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494c2c4-82b5-4c19-a876-cb30addc2562 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:39.339784 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 333/651] 10.222.0.20 () {56 vars in 1137 bytes} [Tue Jul 26 12:38:39 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:39.340607 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494c2c4-82b5-4c19-a876-cb30addc2562 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:39.354787 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205974fe-346a-465b-b85b-f4cd410dec66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:39.355832 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205974fe-346a-465b-b85b-f4cd410dec66 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:39.356315 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205974fe-346a-465b-b85b-f4cd410dec66 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:39.356902 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205974fe-346a-465b-b85b-f4cd410dec66 None admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:39.357977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205974fe-346a-465b-b85b-f4cd410dec66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:39.358498 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205974fe-346a-465b-b85b-f4cd410dec66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:39.360411 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205974fe-346a-465b-b85b-f4cd410dec66 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:39.374582 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 319/652] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:39.375727 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205974fe-346a-465b-b85b-f4cd410dec66 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:39.465886 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494c2c4-82b5-4c19-a876-cb30addc2562 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:39.466805 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:39.482688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:39.493047 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205974fe-346a-465b-b85b-f4cd410dec66 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:39.505800 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe83d0f9-3df4-416c-8daf-fdd2aebc1be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:39.506435 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe83d0f9-3df4-416c-8daf-fdd2aebc1be1 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:39.506711 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe83d0f9-3df4-416c-8daf-fdd2aebc1be1 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:39.506980 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe83d0f9-3df4-416c-8daf-fdd2aebc1be1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:39.507893 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe83d0f9-3df4-416c-8daf-fdd2aebc1be1 None admin] RBAC: Adding query filter params (role.id=9c68629183974be9bfb1ed95f69912ee, scope.project.id=644278e7e8934b37a1e7ab20d35764cd, user.id=0f5d6f3c12244c0faf26048513dc49e7) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:39.508311 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe83d0f9-3df4-416c-8daf-fdd2aebc1be1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:39.509738 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe83d0f9-3df4-416c-8daf-fdd2aebc1be1 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:39.523678 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 334/653] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:38:39 2022] GET /identity/v3/role_assignments?user.id=0f5d6f3c12244c0faf26048513dc49e7&scope.project.id=644278e7e8934b37a1e7ab20d35764cd&role.id=9c68629183974be9bfb1ed95f69912ee => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:39.523678 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe83d0f9-3df4-416c-8daf-fdd2aebc1be1 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:39.542855 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee7437-80f2-4c58-96ac-9edf52a2fee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:39.543495 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee7437-80f2-4c58-96ac-9edf52a2fee2 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:39.543826 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee7437-80f2-4c58-96ac-9edf52a2fee2 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:39.544106 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee7437-80f2-4c58-96ac-9edf52a2fee2 None admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:39.545002 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ee7437-80f2-4c58-96ac-9edf52a2fee2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:39.545685 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ee7437-80f2-4c58-96ac-9edf52a2fee2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:39.547137 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ee7437-80f2-4c58-96ac-9edf52a2fee2 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:39.564100 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 320/654] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:39.565089 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee7437-80f2-4c58-96ac-9edf52a2fee2 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:39.666591 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe83d0f9-3df4-416c-8daf-fdd2aebc1be1 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:39.690438 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee7437-80f2-4c58-96ac-9edf52a2fee2 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:39.722684 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2278b6-41e5-45f2-84af-25c13c14d1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:39.723447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2278b6-41e5-45f2-84af-25c13c14d1e8 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:39.723767 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2278b6-41e5-45f2-84af-25c13c14d1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:39.724085 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2278b6-41e5-45f2-84af-25c13c14d1e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:39.738881 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa7653-1dce-49f9-b454-1ed575038159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:39.739748 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa7653-1dce-49f9-b454-1ed575038159 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:39.740081 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa7653-1dce-49f9-b454-1ed575038159 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:39.740984 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa7653-1dce-49f9-b454-1ed575038159 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:39.755266 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aa7653-1dce-49f9-b454-1ed575038159 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:39.756851 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aa7653-1dce-49f9-b454-1ed575038159 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:39.758939 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:39.772213 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-08aa7653-1dce-49f9-b454-1ed575038159 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 12:38:39.772213 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:39.772213 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:39.772213 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:39.772213 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:39.772213 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:39.772213 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:39.772213 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:39.772213 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:39.772213 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:39.772213 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:39.772213 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:39.772213 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 12:38:39.772213 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 12:38:39.772213 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 12:38:39.772213 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 12:38:39.773276 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:39.773276 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:39.773276 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 12:38:39.773276 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:39.773276 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 12:38:39.773276 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:39.773276 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:39.773276 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:39.773276 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:39.773276 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:39.773276 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:39.773276 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:39.773276 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:39.773276 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:39.773276 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:39.773276 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:39.773276 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:39.773276 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:39.774207 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:39.774207 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:39.774207 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 12:38:39.774207 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 12:38:39.774207 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 12:38:39.774207 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 12:38:39.774207 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 12:38:39.774207 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 12:38:39.774207 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 12:38:39.774207 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:39.774207 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 321/655] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:38:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:39.774207 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa7653-1dce-49f9-b454-1ed575038159 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:39.801275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:39.835487 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2278b6-41e5-45f2-84af-25c13c14d1e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:39.837160 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2278b6-41e5-45f2-84af-25c13c14d1e8 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:39.909730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa7653-1dce-49f9-b454-1ed575038159 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:39.950228 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c38d32-f9fe-4a05-a9ec-0372915e1d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:39.950875 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c38d32-f9fe-4a05-a9ec-0372915e1d39 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:39.951363 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c38d32-f9fe-4a05-a9ec-0372915e1d39 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:39.951472 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c38d32-f9fe-4a05-a9ec-0372915e1d39 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:39.955973 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 335/656] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:38:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 12:38:39.957052 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2278b6-41e5-45f2-84af-25c13c14d1e8 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:39.959365 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c38d32-f9fe-4a05-a9ec-0372915e1d39 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:39.961073 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c38d32-f9fe-4a05-a9ec-0372915e1d39 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:39.973005 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-d6c38d32-f9fe-4a05-a9ec-0372915e1d39 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 12:38:39.973005 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:39.973005 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:39.973005 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:39.973005 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:39.973005 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:39.973005 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:39.973005 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:39.973005 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:39.973005 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:39.973005 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:39.973005 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:39.973005 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 12:38:39.973005 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 12:38:39.973005 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 12:38:39.973005 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 12:38:39.973963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:39.973963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:39.973963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 12:38:39.973963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:39.973963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 12:38:39.973963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:39.973963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:39.973963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:39.973963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:39.973963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:39.973963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:39.973963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:39.973963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:39.973963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:39.973963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:39.973963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:39.973963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:39.973963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:39.975413 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:39.975413 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:39.975413 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 12:38:39.975413 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 12:38:39.975413 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 12:38:39.975413 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 12:38:39.975413 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 12:38:39.975413 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 12:38:39.975413 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 12:38:39.975413 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:39.975413 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 322/657] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 26 12:38:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:39.976055 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72523c8-ff41-40b3-be31-f7a7662f3d77 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:39.976616 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72523c8-ff41-40b3-be31-f7a7662f3d77 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:39.976839 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72523c8-ff41-40b3-be31-f7a7662f3d77 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:39.979526 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 323/658] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:39.979526 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72523c8-ff41-40b3-be31-f7a7662f3d77 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:40.049491 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:40.050605 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:40.087012 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2278b6-41e5-45f2-84af-25c13c14d1e8 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:40.095484 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72523c8-ff41-40b3-be31-f7a7662f3d77 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:40.128892 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8aa63e-aee9-460a-920e-69fb2d7aa4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:40.129514 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8aa63e-aee9-460a-920e-69fb2d7aa4b7 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:40.129797 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8aa63e-aee9-460a-920e-69fb2d7aa4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:40.130150 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8aa63e-aee9-460a-920e-69fb2d7aa4b7 None admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:40.130921 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8aa63e-aee9-460a-920e-69fb2d7aa4b7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:40.131217 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8aa63e-aee9-460a-920e-69fb2d7aa4b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:40.132693 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8aa63e-aee9-460a-920e-69fb2d7aa4b7 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:40.134574 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b049d-65ca-486b-84be-15617861921c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:40.135221 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b049d-65ca-486b-84be-15617861921c None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:40.136334 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b049d-65ca-486b-84be-15617861921c None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:40.136612 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b049d-65ca-486b-84be-15617861921c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:40.148629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770b049d-65ca-486b-84be-15617861921c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:40.150609 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770b049d-65ca-486b-84be-15617861921c None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:40.153502 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 336/659] 10.222.0.20 () {56 vars in 1137 bytes} [Tue Jul 26 12:38:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:40.154208 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8aa63e-aee9-460a-920e-69fb2d7aa4b7 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:40.172177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-770b049d-65ca-486b-84be-15617861921c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:40.172177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:40.172177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:40.172177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:40.172177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:40.172177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:40.172177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:40.172177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:40.172177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:40.172177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:40.172177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:40.172177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:40.172177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:40.172177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:40.172177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:40.173290 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:40.173290 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:40.173290 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:40.173290 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:40.173290 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:40.173290 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:40.173290 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:40.173290 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:40.173290 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:40.173290 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:40.173290 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:40.173290 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:40.173290 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:40.173290 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:40.173290 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:40.173290 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:40.173290 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:40.174268 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:40.174268 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:40.174268 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:40.174268 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:40.177976 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 324/660] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:40.179453 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c132f72-cbf5-4a7f-94bf-190c5f8505df None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:40.179736 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c132f72-cbf5-4a7f-94bf-190c5f8505df None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:40.180021 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c132f72-cbf5-4a7f-94bf-190c5f8505df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:40.198030 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-8c132f72-cbf5-4a7f-94bf-190c5f8505df None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:40.199240 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-8c132f72-cbf5-4a7f-94bf-190c5f8505df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:40.268392 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8aa63e-aee9-460a-920e-69fb2d7aa4b7 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:40.269139 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 325/661] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:38:40.269988 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c132f72-cbf5-4a7f-94bf-190c5f8505df None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:40.312049 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d48884-5c09-4c86-9ba9-2468574d5b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:40.312947 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d48884-5c09-4c86-9ba9-2468574d5b26 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:40.313440 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d48884-5c09-4c86-9ba9-2468574d5b26 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:40.313998 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d48884-5c09-4c86-9ba9-2468574d5b26 None admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:40.315113 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d48884-5c09-4c86-9ba9-2468574d5b26 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:40.315425 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d48884-5c09-4c86-9ba9-2468574d5b26 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:40.316892 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d48884-5c09-4c86-9ba9-2468574d5b26 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:40.318607 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:40.323244 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:40.341899 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 337/662] 10.222.0.20 () {56 vars in 1139 bytes} [Tue Jul 26 12:38:40 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:40.341899 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d48884-5c09-4c86-9ba9-2468574d5b26 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:40.392913 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c132f72-cbf5-4a7f-94bf-190c5f8505df None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:40.433472 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b29b2-d436-4664-87cc-a78a1644c5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:40.434495 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b29b2-d436-4664-87cc-a78a1644c5fa None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:40.435025 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b29b2-d436-4664-87cc-a78a1644c5fa None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:40.435493 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b29b2-d436-4664-87cc-a78a1644c5fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:40.443895 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d48884-5c09-4c86-9ba9-2468574d5b26 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:40.478909 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb457cbb-54f4-48da-8c1b-18b2066fdca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:40.479619 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb457cbb-54f4-48da-8c1b-18b2066fdca7 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:40.479962 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb457cbb-54f4-48da-8c1b-18b2066fdca7 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:40.480300 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb457cbb-54f4-48da-8c1b-18b2066fdca7 None admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:40.480991 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb457cbb-54f4-48da-8c1b-18b2066fdca7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:40.481292 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb457cbb-54f4-48da-8c1b-18b2066fdca7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:40.482644 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb457cbb-54f4-48da-8c1b-18b2066fdca7 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:40.495045 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 338/663] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:40.495878 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb457cbb-54f4-48da-8c1b-18b2066fdca7 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:40.518007 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2b29b2-d436-4664-87cc-a78a1644c5fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:40.520154 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2b29b2-d436-4664-87cc-a78a1644c5fa None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:40.552687 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:40.567223 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:40.603480 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb457cbb-54f4-48da-8c1b-18b2066fdca7 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:40.636632 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 326/664] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:38:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 12:38:40.636632 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b29b2-d436-4664-87cc-a78a1644c5fa None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:40.646714 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d117150-c884-42b3-b665-702baf788511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:40.647686 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d117150-c884-42b3-b665-702baf788511 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:40.647975 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d117150-c884-42b3-b665-702baf788511 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:40.648245 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d117150-c884-42b3-b665-702baf788511 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:40.658721 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d117150-c884-42b3-b665-702baf788511 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:38:40.665348 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-2d117150-c884-42b3-b665-702baf788511 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 12:38:40.665348 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:40.665348 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:40.665348 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:40.665348 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:40.665348 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:40.665348 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:40.665348 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:40.665348 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:40.665348 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:40.665348 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:40.665348 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:40.665348 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:38:40.665348 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:38:40.665348 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:38:40.665348 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:38:40.666270 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:38:40.666270 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:38:40.666270 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:38:40.666270 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:38:40.666270 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:40.666270 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:40.666270 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:40.666270 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:40.666270 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:40.666270 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:40.666270 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:40.666270 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:40.666270 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:40.666270 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:40.666270 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:40.666270 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:40.667100 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:40.667100 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:40.667100 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:40.667100 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:40.667100 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:38:40.667100 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:38:40.667100 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:38:40.667100 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:38:40.667100 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:38:40.667100 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:38:40.667100 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 12:38:40.667100 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:40.667100 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 339/665] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 26 12:38:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:40.668330 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d117150-c884-42b3-b665-702baf788511 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:40.746953 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b29b2-d436-4664-87cc-a78a1644c5fa None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:40.781812 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d117150-c884-42b3-b665-702baf788511 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:40.786844 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2131c399-6725-4e74-934e-8d8fe60c41fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:40.787525 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131c399-6725-4e74-934e-8d8fe60c41fd None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:40.787837 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131c399-6725-4e74-934e-8d8fe60c41fd None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:40.788111 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131c399-6725-4e74-934e-8d8fe60c41fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:40.797386 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:40.803556 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:40.822475 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd18cf-77a8-4071-837b-97c4dac6d78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:40.823058 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd18cf-77a8-4071-837b-97c4dac6d78d None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:40.823406 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd18cf-77a8-4071-837b-97c4dac6d78d None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:40.823796 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd18cf-77a8-4071-837b-97c4dac6d78d None admin] PATH_INFO: `/v3/projects/service` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:40.831292 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccd18cf-77a8-4071-837b-97c4dac6d78d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:40.832824 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccd18cf-77a8-4071-837b-97c4dac6d78d None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:40.845014 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-6ccd18cf-77a8-4071-837b-97c4dac6d78d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:40.845014 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:40.845014 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:40.845014 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:40.845014 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:40.845014 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:40.845014 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:40.845014 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:40.845014 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:40.845014 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:40.845014 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:40.845014 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:40.845014 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 12:38:40.845014 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 12:38:40.845014 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 12:38:40.846119 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 12:38:40.846119 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:40.846119 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:40.846119 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:40.846119 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:40.846119 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:40.846119 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:40.846119 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:40.846119 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:40.846119 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:40.846119 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:40.846119 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:40.846119 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:40.846119 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:40.846119 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:40.846119 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:40.846119 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:40.847249 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 12:38:40.847249 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 12:38:40.847249 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 12:38:40.847249 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 12:38:40.847249 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 12:38:40.847249 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 12:38:40.847249 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:40.847249 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:40.850974 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 340/666] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:38:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:40.851880 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd18cf-77a8-4071-837b-97c4dac6d78d None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:40.874316 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2131c399-6725-4e74-934e-8d8fe60c41fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:40.876184 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2131c399-6725-4e74-934e-8d8fe60c41fd None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:40.965922 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd18cf-77a8-4071-837b-97c4dac6d78d None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:40.993082 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 327/667] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:38:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 12:38:40.994366 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2131c399-6725-4e74-934e-8d8fe60c41fd None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:41.009751 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd15e9da-b180-4901-9fe8-686f139d31cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:41.010388 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15e9da-b180-4901-9fe8-686f139d31cc None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:41.010674 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15e9da-b180-4901-9fe8-686f139d31cc None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:41.010952 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15e9da-b180-4901-9fe8-686f139d31cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:41.022017 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd15e9da-b180-4901-9fe8-686f139d31cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:41.023478 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd15e9da-b180-4901-9fe8-686f139d31cc None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:41.039358 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-fd15e9da-b180-4901-9fe8-686f139d31cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:41.039358 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:41.039358 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:41.039358 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:41.039358 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:41.039358 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:41.039358 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:41.039358 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:41.039358 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:41.039358 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:41.039358 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:41.039358 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:41.039358 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:41.039358 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:41.039358 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:41.040275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:41.040275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:41.040275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:41.040275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:41.040275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:41.040275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:41.040275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:41.040275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:41.040275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:41.040275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:41.040275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:41.040275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:41.040275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:41.040275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:41.040275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:41.040275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:41.040275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:41.041170 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:41.041170 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:41.041170 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:41.041170 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:41.042085 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:41.044309 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 341/668] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:41.045383 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd15e9da-b180-4901-9fe8-686f139d31cc None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:41.075711 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:41.110631 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2131c399-6725-4e74-934e-8d8fe60c41fd None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:41.151758 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252de397-3009-4fe1-bdcb-d32ccb2b84a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:41.152378 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252de397-3009-4fe1-bdcb-d32ccb2b84a5 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:41.152819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252de397-3009-4fe1-bdcb-d32ccb2b84a5 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:41.153097 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252de397-3009-4fe1-bdcb-d32ccb2b84a5 None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:41.154270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252de397-3009-4fe1-bdcb-d32ccb2b84a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:41.154705 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252de397-3009-4fe1-bdcb-d32ccb2b84a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:41.156590 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252de397-3009-4fe1-bdcb-d32ccb2b84a5 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:41.166393 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd15e9da-b180-4901-9fe8-686f139d31cc None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:41.175823 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 328/669] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Jul 26 12:38:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:41.175823 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252de397-3009-4fe1-bdcb-d32ccb2b84a5 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:41.207053 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75274660-c7f8-47a2-a80b-3913c50f6ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:41.208082 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75274660-c7f8-47a2-a80b-3913c50f6ba5 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:41.208568 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75274660-c7f8-47a2-a80b-3913c50f6ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:41.209187 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75274660-c7f8-47a2-a80b-3913c50f6ba5 None admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:41.210154 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75274660-c7f8-47a2-a80b-3913c50f6ba5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:41.210677 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75274660-c7f8-47a2-a80b-3913c50f6ba5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:41.212328 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75274660-c7f8-47a2-a80b-3913c50f6ba5 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:41.225955 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 342/670] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 26 12:38:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:41.227130 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75274660-c7f8-47a2-a80b-3913c50f6ba5 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:41.287622 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252de397-3009-4fe1-bdcb-d32ccb2b84a5 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:41.289322 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:41.326645 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae022fd-fc90-4f80-a9bf-caad92bd869a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:41.327192 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae022fd-fc90-4f80-a9bf-caad92bd869a None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:41.327475 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae022fd-fc90-4f80-a9bf-caad92bd869a None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:41.327780 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae022fd-fc90-4f80-a9bf-caad92bd869a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:41.328632 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:41.339773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae022fd-fc90-4f80-a9bf-caad92bd869a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:41.341308 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae022fd-fc90-4f80-a9bf-caad92bd869a None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:41.344485 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75274660-c7f8-47a2-a80b-3913c50f6ba5 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:41.357446 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-bae022fd-fc90-4f80-a9bf-caad92bd869a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:41.357446 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:41.357446 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:41.357446 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:41.357446 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:41.357446 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:41.357446 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:41.357446 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:41.357446 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:41.357446 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:41.357446 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:41.357446 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:41.357446 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:41.357446 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:41.357446 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:41.358369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:41.358369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:41.358369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:41.358369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:41.358369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:41.358369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:41.358369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:41.358369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:41.358369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:41.358369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:41.358369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:41.358369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:41.358369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:41.358369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:41.358369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:41.358369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:41.358369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:41.359271 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:41.359271 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:41.359271 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:41.359271 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:41.361054 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 329/671] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:41.361979 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae022fd-fc90-4f80-a9bf-caad92bd869a None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:41.386102 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cfbc7c-be0a-4d86-ae94-857759ed9ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:41.386725 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cfbc7c-be0a-4d86-ae94-857759ed9ec5 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:41.387008 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cfbc7c-be0a-4d86-ae94-857759ed9ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:41.387285 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cfbc7c-be0a-4d86-ae94-857759ed9ec5 None admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:41.388027 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cfbc7c-be0a-4d86-ae94-857759ed9ec5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:41.388334 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cfbc7c-be0a-4d86-ae94-857759ed9ec5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:41.389716 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cfbc7c-be0a-4d86-ae94-857759ed9ec5 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:41.402052 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 343/672] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:41.402947 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cfbc7c-be0a-4d86-ae94-857759ed9ec5 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:41.481667 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae022fd-fc90-4f80-a9bf-caad92bd869a None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:41.519462 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae61b7-a6a5-4920-9c95-59c8dd701770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:41.520140 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae61b7-a6a5-4920-9c95-59c8dd701770 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:41.520533 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae61b7-a6a5-4920-9c95-59c8dd701770 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:41.520843 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae61b7-a6a5-4920-9c95-59c8dd701770 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:41.532784 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae61b7-a6a5-4920-9c95-59c8dd701770 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:41.534343 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae61b7-a6a5-4920-9c95-59c8dd701770 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:41.544606 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cfbc7c-be0a-4d86-ae94-857759ed9ec5 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:41.549379 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-5fae61b7-a6a5-4920-9c95-59c8dd701770 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:41.549379 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:41.549379 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:41.549379 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:41.549379 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:41.549379 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:41.549379 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:41.549379 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:41.549379 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:41.549379 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:41.549379 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:41.549379 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:41.549379 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:41.549379 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:41.549379 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:41.550258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:41.550258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:41.550258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:41.550258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:41.550258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:41.550258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:41.550258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:41.550258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:41.550258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:41.550258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:41.550258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:41.550258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:41.550258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:41.550258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:41.550258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:41.550258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:41.550258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:41.556735 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:41.556735 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:41.556735 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:41.556735 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:41.556735 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 330/673] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:41.556735 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae61b7-a6a5-4920-9c95-59c8dd701770 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:41.565050 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:41.589982 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d0047b-3be1-46b8-b8c0-89af8820e417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:41.591053 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0047b-3be1-46b8-b8c0-89af8820e417 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:41.591483 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0047b-3be1-46b8-b8c0-89af8820e417 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:41.591962 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0047b-3be1-46b8-b8c0-89af8820e417 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:41.593133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d0047b-3be1-46b8-b8c0-89af8820e417 None admin] RBAC: Adding query filter params (role.id=9c68629183974be9bfb1ed95f69912ee, scope.project.id=644278e7e8934b37a1e7ab20d35764cd, user.id=d2b5853838dc40db89c46038c0606107) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:41.594334 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d0047b-3be1-46b8-b8c0-89af8820e417 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:41.595970 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d0047b-3be1-46b8-b8c0-89af8820e417 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:41.603586 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 344/674] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:38:41 2022] GET /identity/v3/role_assignments?user.id=d2b5853838dc40db89c46038c0606107&scope.project.id=644278e7e8934b37a1e7ab20d35764cd&role.id=9c68629183974be9bfb1ed95f69912ee => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:41.604738 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d0047b-3be1-46b8-b8c0-89af8820e417 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:41.623262 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:41.669481 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae61b7-a6a5-4920-9c95-59c8dd701770 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:41.712312 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c06e315-9340-404e-a2d7-8b0d22bd5ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:41.712980 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c06e315-9340-404e-a2d7-8b0d22bd5ec7 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:41.713269 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c06e315-9340-404e-a2d7-8b0d22bd5ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:41.713554 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c06e315-9340-404e-a2d7-8b0d22bd5ec7 None admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:41.714243 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c06e315-9340-404e-a2d7-8b0d22bd5ec7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:41.714541 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c06e315-9340-404e-a2d7-8b0d22bd5ec7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:41.716004 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c06e315-9340-404e-a2d7-8b0d22bd5ec7 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:41.732229 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 331/675] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:41.732229 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c06e315-9340-404e-a2d7-8b0d22bd5ec7 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:41.748247 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d0047b-3be1-46b8-b8c0-89af8820e417 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:41.789139 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17f78da-1f98-48ea-aca3-c1f52a72c4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:41.789788 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f78da-1f98-48ea-aca3-c1f52a72c4d2 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:41.790070 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f78da-1f98-48ea-aca3-c1f52a72c4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:41.790342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f78da-1f98-48ea-aca3-c1f52a72c4d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:41.801400 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17f78da-1f98-48ea-aca3-c1f52a72c4d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:41.803402 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17f78da-1f98-48ea-aca3-c1f52a72c4d2 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:41.817814 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-c17f78da-1f98-48ea-aca3-c1f52a72c4d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:41.817814 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:41.817814 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:41.817814 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:41.817814 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:41.817814 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:41.817814 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:41.817814 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:41.817814 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:41.817814 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:41.817814 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:41.817814 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:41.817814 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 12:38:41.817814 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 12:38:41.817814 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 12:38:41.819340 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 12:38:41.819340 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:41.819340 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:41.819340 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:41.819340 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:41.819340 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:41.819340 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:41.819340 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:41.819340 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:41.819340 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:41.819340 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:41.819340 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:41.819340 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:41.819340 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:41.819340 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:41.819340 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:41.819340 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:41.820300 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 12:38:41.820300 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 12:38:41.820300 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 12:38:41.820300 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 12:38:41.820300 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 12:38:41.820300 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 12:38:41.820300 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:41.820300 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:41.820300 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 345/676] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:38:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:41.820769 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4060f3-f3e2-42d2-affc-9891236cb2fd None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:41.820890 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4060f3-f3e2-42d2-affc-9891236cb2fd None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:41.821165 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4060f3-f3e2-42d2-affc-9891236cb2fd None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:41.822402 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 346/677] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:41.823141 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4060f3-f3e2-42d2-affc-9891236cb2fd None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:41.832697 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:41.876835 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c06e315-9340-404e-a2d7-8b0d22bd5ec7 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:41.904551 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:41.913626 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729156a7-fead-418c-b9a9-4570b00d4236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:41.914255 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729156a7-fead-418c-b9a9-4570b00d4236 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:41.914926 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729156a7-fead-418c-b9a9-4570b00d4236 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:41.914926 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729156a7-fead-418c-b9a9-4570b00d4236 None admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:41.915529 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729156a7-fead-418c-b9a9-4570b00d4236 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:41.915865 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729156a7-fead-418c-b9a9-4570b00d4236 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:41.917237 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729156a7-fead-418c-b9a9-4570b00d4236 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:41.929556 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 332/678] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:41.930386 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729156a7-fead-418c-b9a9-4570b00d4236 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:41.950978 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4060f3-f3e2-42d2-affc-9891236cb2fd None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:41.990207 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9997ff9-08b3-44b7-959e-c5d762837049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:41.990905 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9997ff9-08b3-44b7-959e-c5d762837049 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:41.991183 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9997ff9-08b3-44b7-959e-c5d762837049 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:41.991450 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9997ff9-08b3-44b7-959e-c5d762837049 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:41.998958 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9997ff9-08b3-44b7-959e-c5d762837049 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:42.000435 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9997ff9-08b3-44b7-959e-c5d762837049 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:42.012452 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-e9997ff9-08b3-44b7-959e-c5d762837049 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:42.012452 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:42.012452 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:42.012452 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:42.012452 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:42.012452 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:42.012452 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:42.012452 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:42.012452 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:42.012452 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:42.012452 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:42.012452 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:42.012452 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 12:38:42.012452 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 12:38:42.012452 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 12:38:42.013401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 12:38:42.013401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:42.013401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:42.013401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:42.013401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:42.013401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:42.013401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:42.013401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:42.013401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:42.013401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:42.013401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:42.013401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:42.013401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:42.013401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:42.013401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:42.013401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:42.013401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:42.014266 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 12:38:42.014266 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 12:38:42.014266 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 12:38:42.014266 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 12:38:42.014266 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 12:38:42.014266 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 12:38:42.014266 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:42.014266 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:42.018191 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 347/679] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:38:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:42.019415 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa980850-8b32-4540-9630-67a076b66056 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:42.019743 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa980850-8b32-4540-9630-67a076b66056 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:42.020020 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa980850-8b32-4540-9630-67a076b66056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:42.039134 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-fa980850-8b32-4540-9630-67a076b66056 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:42.040359 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-fa980850-8b32-4540-9630-67a076b66056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:42.044173 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729156a7-fead-418c-b9a9-4570b00d4236 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:42.074340 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:42.083687 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc0443-7286-4cb0-a23d-bed27bac4e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:42.084649 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc0443-7286-4cb0-a23d-bed27bac4e81 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:42.084810 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc0443-7286-4cb0-a23d-bed27bac4e81 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:42.084978 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc0443-7286-4cb0-a23d-bed27bac4e81 None admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:42.085709 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc0443-7286-4cb0-a23d-bed27bac4e81 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:42.086012 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc0443-7286-4cb0-a23d-bed27bac4e81 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:42.087433 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc0443-7286-4cb0-a23d-bed27bac4e81 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:42.102256 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 333/680] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 26 12:38:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:42.109579 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc0443-7286-4cb0-a23d-bed27bac4e81 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:42.117612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 348/681] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:38:42.125352 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa980850-8b32-4540-9630-67a076b66056 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:42.202196 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:42.225961 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc0443-7286-4cb0-a23d-bed27bac4e81 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:42.250291 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa980850-8b32-4540-9630-67a076b66056 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:42.266282 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e974a3b-77e0-4c13-b85a-ca1debd464e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:42.267147 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e974a3b-77e0-4c13-b85a-ca1debd464e7 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:42.267577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e974a3b-77e0-4c13-b85a-ca1debd464e7 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:42.268049 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e974a3b-77e0-4c13-b85a-ca1debd464e7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:42.275760 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e974a3b-77e0-4c13-b85a-ca1debd464e7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:42.277398 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e974a3b-77e0-4c13-b85a-ca1debd464e7 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:42.286663 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab567c-871d-49e8-b61e-d1483bb97b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:42.287799 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab567c-871d-49e8-b61e-d1483bb97b7c None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:42.288077 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab567c-871d-49e8-b61e-d1483bb97b7c None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:42.288347 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab567c-871d-49e8-b61e-d1483bb97b7c None admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:42.289087 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab567c-871d-49e8-b61e-d1483bb97b7c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:42.289389 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab567c-871d-49e8-b61e-d1483bb97b7c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:42.290814 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab567c-871d-49e8-b61e-d1483bb97b7c None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:42.291799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-9e974a3b-77e0-4c13-b85a-ca1debd464e7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 12:38:42.291799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:42.291799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:42.291799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:42.291799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:42.291799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:42.291799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:42.291799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:42.291799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:42.291799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:42.291799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:42.291799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:42.291799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 12:38:42.291799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 12:38:42.291799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 12:38:42.291799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 12:38:42.292923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:42.292923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:42.292923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 12:38:42.292923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:42.292923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 12:38:42.292923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:42.292923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:42.292923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:42.292923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:42.292923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:42.292923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:42.292923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:42.292923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:42.292923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:42.292923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:42.292923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:42.292923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:42.292923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:42.293972 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:42.293972 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:42.293972 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 12:38:42.293972 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 12:38:42.293972 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 12:38:42.293972 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 12:38:42.293972 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 12:38:42.293972 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 12:38:42.293972 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 12:38:42.293972 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:42.295872 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 334/682] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 26 12:38:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:42.296788 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e974a3b-77e0-4c13-b85a-ca1debd464e7 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:42.306862 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 349/683] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 26 12:38:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:42.307716 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab567c-871d-49e8-b61e-d1483bb97b7c None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:42.352091 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:42.410815 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e974a3b-77e0-4c13-b85a-ca1debd464e7 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:42.417702 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab567c-871d-49e8-b61e-d1483bb97b7c None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:42.436558 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:42.453080 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef2bd0-e530-4c09-8612-2c0e7e784b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:42.453722 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef2bd0-e530-4c09-8612-2c0e7e784b6f None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:42.454013 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef2bd0-e530-4c09-8612-2c0e7e784b6f None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:42.454321 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef2bd0-e530-4c09-8612-2c0e7e784b6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:42.455095 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ef2bd0-e530-4c09-8612-2c0e7e784b6f None admin] RBAC: Adding query filter params (role.id=9c68629183974be9bfb1ed95f69912ee, scope.project.id=644278e7e8934b37a1e7ab20d35764cd, user.id=9f8ca5e019dd4b9090cee2b2c1d8e15d) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:42.455417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ef2bd0-e530-4c09-8612-2c0e7e784b6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:42.456113 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0805cce5-b7a6-416b-904f-a250248c871c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:42.456792 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0805cce5-b7a6-416b-904f-a250248c871c None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:42.456943 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ef2bd0-e530-4c09-8612-2c0e7e784b6f None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:42.457068 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0805cce5-b7a6-416b-904f-a250248c871c None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:42.457321 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0805cce5-b7a6-416b-904f-a250248c871c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:42.472104 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 335/684] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:38:42 2022] GET /identity/v3/role_assignments?user.id=9f8ca5e019dd4b9090cee2b2c1d8e15d&scope.project.id=644278e7e8934b37a1e7ab20d35764cd&role.id=9c68629183974be9bfb1ed95f69912ee => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:42.472104 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef2bd0-e530-4c09-8612-2c0e7e784b6f None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:42.480318 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0805cce5-b7a6-416b-904f-a250248c871c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:38:42.486083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-0805cce5-b7a6-416b-904f-a250248c871c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:38:42.486083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:42.486083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:42.486083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:42.486083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:42.486083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:42.486083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:42.486083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:42.486083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:42.486083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:42.486083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:42.486083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:42.486083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:38:42.486083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:38:42.486083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:38:42.486083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:38:42.487121 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:38:42.487121 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:38:42.487121 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:38:42.487121 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:38:42.487121 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:42.487121 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:42.487121 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:42.487121 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:42.487121 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:42.487121 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:42.487121 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:42.487121 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:42.487121 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:42.487121 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:42.487121 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:42.487121 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:42.488613 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:42.488613 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:42.488613 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:42.488613 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:42.488613 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:38:42.488613 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:38:42.488613 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:38:42.488613 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:38:42.488613 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:38:42.488613 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:38:42.488613 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:38:42.488613 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:42.488613 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 350/685] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 26 12:38:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:42.488613 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0805cce5-b7a6-416b-904f-a250248c871c None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:42.604292 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0805cce5-b7a6-416b-904f-a250248c871c None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:42.606104 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef2bd0-e530-4c09-8612-2c0e7e784b6f None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:42.622153 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:42.653422 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2427ff60-0b26-4f99-a8f6-184b9e3d20d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:42.654433 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427ff60-0b26-4f99-a8f6-184b9e3d20d1 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:42.654916 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427ff60-0b26-4f99-a8f6-184b9e3d20d1 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:42.655412 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427ff60-0b26-4f99-a8f6-184b9e3d20d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:42.656380 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2427ff60-0b26-4f99-a8f6-184b9e3d20d1 None admin] RBAC: Adding query filter params (role.id=9c68629183974be9bfb1ed95f69912ee, scope.project.id=644278e7e8934b37a1e7ab20d35764cd, user.id=6830350c6eba4610bbe9ab1dc31f5939) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:42.656747 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2427ff60-0b26-4f99-a8f6-184b9e3d20d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:42.658256 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2427ff60-0b26-4f99-a8f6-184b9e3d20d1 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:42.660937 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1647cf4-ff82-4010-8847-f189970aa742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:42.661561 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1647cf4-ff82-4010-8847-f189970aa742 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:42.661842 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1647cf4-ff82-4010-8847-f189970aa742 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:42.662204 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1647cf4-ff82-4010-8847-f189970aa742 None admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:42.663020 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1647cf4-ff82-4010-8847-f189970aa742 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:42.663334 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1647cf4-ff82-4010-8847-f189970aa742 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:42.664764 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1647cf4-ff82-4010-8847-f189970aa742 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:42.665885 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 351/686] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:38:42 2022] GET /identity/v3/role_assignments?user.id=6830350c6eba4610bbe9ab1dc31f5939&scope.project.id=644278e7e8934b37a1e7ab20d35764cd&role.id=9c68629183974be9bfb1ed95f69912ee => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:42.666916 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2427ff60-0b26-4f99-a8f6-184b9e3d20d1 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:42.686473 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 336/687] 10.222.0.20 () {56 vars in 1135 bytes} [Tue Jul 26 12:38:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:42.693297 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1647cf4-ff82-4010-8847-f189970aa742 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:42.719329 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:42.808296 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2427ff60-0b26-4f99-a8f6-184b9e3d20d1 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:42.841269 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1647cf4-ff82-4010-8847-f189970aa742 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:42.858823 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9566ac-0262-4c6a-8fab-8f18e60a4046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:42.859466 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9566ac-0262-4c6a-8fab-8f18e60a4046 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:42.859779 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9566ac-0262-4c6a-8fab-8f18e60a4046 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:42.860059 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9566ac-0262-4c6a-8fab-8f18e60a4046 None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:42.860935 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9566ac-0262-4c6a-8fab-8f18e60a4046 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:42.861255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9566ac-0262-4c6a-8fab-8f18e60a4046 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:42.862828 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9566ac-0262-4c6a-8fab-8f18e60a4046 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:42.883008 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a8aa14-0a54-4185-b2c0-1f313fe4f1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:42.883709 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a8aa14-0a54-4185-b2c0-1f313fe4f1a6 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:42.883996 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a8aa14-0a54-4185-b2c0-1f313fe4f1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:42.884100 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 352/688] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:38:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:42.884270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a8aa14-0a54-4185-b2c0-1f313fe4f1a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:42.890481 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9566ac-0262-4c6a-8fab-8f18e60a4046 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:42.930409 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:42.973270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a8aa14-0a54-4185-b2c0-1f313fe4f1a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:42.975790 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a8aa14-0a54-4185-b2c0-1f313fe4f1a6 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:43.032818 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9566ac-0262-4c6a-8fab-8f18e60a4046 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:43.035840 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:43.082198 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a44661-0ba5-4a8a-a868-5aaa15894ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:43.083129 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a44661-0ba5-4a8a-a868-5aaa15894ed8 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:43.083410 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a44661-0ba5-4a8a-a868-5aaa15894ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:43.084977 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a44661-0ba5-4a8a-a868-5aaa15894ed8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:43.097501 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a44661-0ba5-4a8a-a868-5aaa15894ed8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:43.099543 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a44661-0ba5-4a8a-a868-5aaa15894ed8 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:43.106545 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 337/689] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:38:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 12:38:43.112857 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a8aa14-0a54-4185-b2c0-1f313fe4f1a6 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:43.126409 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-28a44661-0ba5-4a8a-a868-5aaa15894ed8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:43.126409 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:43.126409 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:43.126409 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:43.126409 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:43.126409 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:43.126409 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:43.126409 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:43.126409 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:43.126409 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:43.126409 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:43.126409 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:43.126409 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:43.126409 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:43.126409 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:43.128697 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:43.128697 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:43.128697 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:43.128697 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:43.128697 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:43.128697 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:43.128697 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:43.128697 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:43.128697 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:43.128697 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:43.128697 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:43.128697 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:43.128697 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:43.128697 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:43.128697 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:43.128697 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:43.128697 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:43.130716 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:43.130716 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:43.130716 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:43.130716 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:43.130716 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 353/690] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:43.139557 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a44661-0ba5-4a8a-a868-5aaa15894ed8 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:43.199212 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:43.269911 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a8aa14-0a54-4185-b2c0-1f313fe4f1a6 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:43.286507 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a44661-0ba5-4a8a-a868-5aaa15894ed8 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:43.321915 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aa04ff-dec4-4a72-8dd9-4f98a73bac73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:43.322544 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa04ff-dec4-4a72-8dd9-4f98a73bac73 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:43.322825 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa04ff-dec4-4a72-8dd9-4f98a73bac73 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:43.323102 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa04ff-dec4-4a72-8dd9-4f98a73bac73 None admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:43.323933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aa04ff-dec4-4a72-8dd9-4f98a73bac73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:43.324484 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aa04ff-dec4-4a72-8dd9-4f98a73bac73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:43.326305 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aa04ff-dec4-4a72-8dd9-4f98a73bac73 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:43.328442 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7e2a6-5b61-4a27-b573-9d64d6887935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:43.329211 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7e2a6-5b61-4a27-b573-9d64d6887935 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:43.329497 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7e2a6-5b61-4a27-b573-9d64d6887935 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:43.329780 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7e2a6-5b61-4a27-b573-9d64d6887935 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:43.346325 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c7e2a6-5b61-4a27-b573-9d64d6887935 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:43.347814 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c7e2a6-5b61-4a27-b573-9d64d6887935 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:43.349362 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 354/691] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:43.354675 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aa04ff-dec4-4a72-8dd9-4f98a73bac73 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:43.388051 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:43.389420 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-e8c7e2a6-5b61-4a27-b573-9d64d6887935 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:43.389420 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:43.389420 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:43.389420 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:43.389420 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:43.389420 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:43.389420 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:43.389420 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:43.389420 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:43.389420 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:43.389420 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:43.389420 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:43.389420 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:43.389420 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:43.389420 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:43.391583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:43.391583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:43.391583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:43.391583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:43.391583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:43.391583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:43.391583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:43.391583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:43.391583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:43.391583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:43.391583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:43.391583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:43.391583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:43.391583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:43.391583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:43.391583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:43.391583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:43.393936 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:43.393936 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:43.393936 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:43.393936 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:43.393936 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 338/692] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:43.396827 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7e2a6-5b61-4a27-b573-9d64d6887935 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:43.495006 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aa04ff-dec4-4a72-8dd9-4f98a73bac73 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:43.530901 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c62e6ce-bc7f-473a-9d93-9c18758e37fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:43.531508 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62e6ce-bc7f-473a-9d93-9c18758e37fb None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:43.531824 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62e6ce-bc7f-473a-9d93-9c18758e37fb None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:43.532099 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62e6ce-bc7f-473a-9d93-9c18758e37fb None admin] PATH_INFO: `/v3/users/nova` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:43.540483 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c62e6ce-bc7f-473a-9d93-9c18758e37fb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:43.541940 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c62e6ce-bc7f-473a-9d93-9c18758e37fb None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:43.554861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-6c62e6ce-bc7f-473a-9d93-9c18758e37fb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 12:38:43.554861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:43.554861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:43.554861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:43.554861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:43.554861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:43.554861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:43.554861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:43.554861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:43.554861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:43.554861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:43.554861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:43.554861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 12:38:43.554861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 12:38:43.554861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 12:38:43.554861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 12:38:43.556822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:43.556822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:43.556822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 12:38:43.556822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:43.556822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 12:38:43.556822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:43.556822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:43.556822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:43.556822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:43.556822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:43.556822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:43.556822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:43.556822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:43.556822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:43.556822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:43.556822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:43.556822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:43.556822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:43.558018 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:43.558018 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:43.558018 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 12:38:43.558018 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 12:38:43.558018 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 12:38:43.558018 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 12:38:43.558018 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 12:38:43.558018 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 12:38:43.558018 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 12:38:43.558018 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:43.558018 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 355/693] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Jul 26 12:38:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:43.564041 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c62e6ce-bc7f-473a-9d93-9c18758e37fb None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:43.652018 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:43.657575 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7e2a6-5b61-4a27-b573-9d64d6887935 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:43.678872 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c62e6ce-bc7f-473a-9d93-9c18758e37fb None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:43.716224 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6636fe4-5bdb-4dba-877f-23ae38798d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:43.716848 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6636fe4-5bdb-4dba-877f-23ae38798d42 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:43.717131 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6636fe4-5bdb-4dba-877f-23ae38798d42 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:43.717408 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6636fe4-5bdb-4dba-877f-23ae38798d42 None admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:43.718205 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6636fe4-5bdb-4dba-877f-23ae38798d42 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:43.719167 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:43.720087 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6636fe4-5bdb-4dba-877f-23ae38798d42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:43.722654 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6636fe4-5bdb-4dba-877f-23ae38798d42 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:43.742844 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 356/694] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Jul 26 12:38:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:38:43.747959 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6636fe4-5bdb-4dba-877f-23ae38798d42 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:43.762821 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da987c0b-315f-4eae-98a4-283d15c76b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:43.763452 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da987c0b-315f-4eae-98a4-283d15c76b01 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:43.763759 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da987c0b-315f-4eae-98a4-283d15c76b01 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:43.764036 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da987c0b-315f-4eae-98a4-283d15c76b01 None admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:43.764738 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da987c0b-315f-4eae-98a4-283d15c76b01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:43.765047 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da987c0b-315f-4eae-98a4-283d15c76b01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:43.766423 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da987c0b-315f-4eae-98a4-283d15c76b01 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:43.795337 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 339/695] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:38:43.801241 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da987c0b-315f-4eae-98a4-283d15c76b01 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:43.922966 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6636fe4-5bdb-4dba-877f-23ae38798d42 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:43.929965 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da987c0b-315f-4eae-98a4-283d15c76b01 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:43.960374 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76effa90-ff01-4189-b783-474ab9a72ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:43.961014 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76effa90-ff01-4189-b783-474ab9a72ca8 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:43.961303 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76effa90-ff01-4189-b783-474ab9a72ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:43.961576 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76effa90-ff01-4189-b783-474ab9a72ca8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:43.964919 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:43.983417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd30952-8b26-4f65-986d-119cc5851999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:43.984688 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd30952-8b26-4f65-986d-119cc5851999 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:43.985509 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd30952-8b26-4f65-986d-119cc5851999 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:43.985970 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd30952-8b26-4f65-986d-119cc5851999 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:43.996296 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd30952-8b26-4f65-986d-119cc5851999 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:43.998814 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd30952-8b26-4f65-986d-119cc5851999 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:44.015451 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-9dd30952-8b26-4f65-986d-119cc5851999 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:44.015451 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:44.015451 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:44.015451 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:44.015451 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:44.015451 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:44.015451 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:44.015451 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:44.015451 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:44.015451 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:44.015451 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:44.015451 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:44.015451 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 12:38:44.015451 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 12:38:44.015451 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 12:38:44.016406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 12:38:44.016406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:44.016406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:44.016406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:44.016406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:44.016406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:44.016406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:44.016406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:44.016406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:44.016406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:44.016406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:44.016406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:44.016406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:44.016406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:44.016406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:44.016406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:44.016406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:44.017296 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 12:38:44.017296 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 12:38:44.017296 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 12:38:44.017296 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 12:38:44.017296 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 12:38:44.017296 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 12:38:44.017296 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:44.017296 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:44.019991 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 340/696] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:38:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:44.021926 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f22bfba-1d31-426e-a6ba-c4ecbeac8f80 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:44.022350 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f22bfba-1d31-426e-a6ba-c4ecbeac8f80 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:44.022775 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f22bfba-1d31-426e-a6ba-c4ecbeac8f80 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:44.024461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 341/697] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:44.026018 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f22bfba-1d31-426e-a6ba-c4ecbeac8f80 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:44.049241 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76effa90-ff01-4189-b783-474ab9a72ca8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:44.050640 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76effa90-ff01-4189-b783-474ab9a72ca8 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:44.073736 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:44.153092 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f22bfba-1d31-426e-a6ba-c4ecbeac8f80 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:44.165924 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 357/698] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:38:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 12:38:44.167400 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6248642d-15ab-4109-a75f-2d3866b7a19d None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:44.167725 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6248642d-15ab-4109-a75f-2d3866b7a19d None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:44.168004 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6248642d-15ab-4109-a75f-2d3866b7a19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:44.188599 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-6248642d-15ab-4109-a75f-2d3866b7a19d None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:44.189802 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-6248642d-15ab-4109-a75f-2d3866b7a19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:44.206366 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61310104-ef11-465c-afa8-b8a8ee4590bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:44.207309 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61310104-ef11-465c-afa8-b8a8ee4590bb None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:44.208013 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61310104-ef11-465c-afa8-b8a8ee4590bb None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:44.208296 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61310104-ef11-465c-afa8-b8a8ee4590bb None admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:44.209120 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61310104-ef11-465c-afa8-b8a8ee4590bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:44.209431 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61310104-ef11-465c-afa8-b8a8ee4590bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:44.210864 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61310104-ef11-465c-afa8-b8a8ee4590bb None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:44.227714 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 342/699] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 26 12:38:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:44.227714 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61310104-ef11-465c-afa8-b8a8ee4590bb None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:44.235270 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:44.341993 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:44.346973 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61310104-ef11-465c-afa8-b8a8ee4590bb None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:44.391685 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d2e8f8-ef18-4e3c-b42d-7de55477db4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:44.392379 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d2e8f8-ef18-4e3c-b42d-7de55477db4d None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:44.392656 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d2e8f8-ef18-4e3c-b42d-7de55477db4d None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:44.392924 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d2e8f8-ef18-4e3c-b42d-7de55477db4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:44.406068 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d2e8f8-ef18-4e3c-b42d-7de55477db4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:44.407776 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d2e8f8-ef18-4e3c-b42d-7de55477db4d None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:44.425605 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-a1d2e8f8-ef18-4e3c-b42d-7de55477db4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:44.425605 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:44.425605 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:44.425605 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:44.425605 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:44.425605 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:44.425605 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:44.425605 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:44.425605 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:44.425605 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:44.425605 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:44.425605 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:44.425605 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:44.425605 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:44.425605 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:44.426577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:44.426577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:44.426577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:44.426577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:44.426577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:44.426577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:44.426577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:44.426577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:44.426577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:44.426577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:44.426577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:44.426577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:44.426577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:44.426577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:44.426577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:44.426577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:44.426577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:44.427458 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:44.427458 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:44.427458 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:44.427458 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:44.431353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 343/700] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:44.432274 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d2e8f8-ef18-4e3c-b42d-7de55477db4d None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:44.452907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 358/701] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:38:44.454089 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6248642d-15ab-4109-a75f-2d3866b7a19d None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:44.586093 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6248642d-15ab-4109-a75f-2d3866b7a19d None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:44.599483 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d2e8f8-ef18-4e3c-b42d-7de55477db4d None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:44.629882 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d69e5e-2cb6-47a2-a362-c1a9f42df310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:44.631488 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d69e5e-2cb6-47a2-a362-c1a9f42df310 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:44.631806 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d69e5e-2cb6-47a2-a362-c1a9f42df310 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:44.632352 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d69e5e-2cb6-47a2-a362-c1a9f42df310 None admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:44.633166 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d69e5e-2cb6-47a2-a362-c1a9f42df310 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:44.633731 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d69e5e-2cb6-47a2-a362-c1a9f42df310 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:44.635815 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d69e5e-2cb6-47a2-a362-c1a9f42df310 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:44.642341 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6743293c-14cf-42a7-8097-540f8e36b7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:44.643067 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743293c-14cf-42a7-8097-540f8e36b7ba None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:44.643459 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743293c-14cf-42a7-8097-540f8e36b7ba None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:44.643955 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743293c-14cf-42a7-8097-540f8e36b7ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:44.644891 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6743293c-14cf-42a7-8097-540f8e36b7ba None admin] RBAC: Adding query filter params (role.id=9c68629183974be9bfb1ed95f69912ee, scope.project.id=644278e7e8934b37a1e7ab20d35764cd, user.id=cd2483f74d7d48b1a7b8ffe32f63abde) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:44.645294 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6743293c-14cf-42a7-8097-540f8e36b7ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:44.646938 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6743293c-14cf-42a7-8097-540f8e36b7ba None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:44.649599 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 359/702] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:44.650430 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d69e5e-2cb6-47a2-a362-c1a9f42df310 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:44.658605 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 344/703] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:38:44 2022] GET /identity/v3/role_assignments?user.id=cd2483f74d7d48b1a7b8ffe32f63abde&scope.project.id=644278e7e8934b37a1e7ab20d35764cd&role.id=9c68629183974be9bfb1ed95f69912ee => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:44.659567 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6743293c-14cf-42a7-8097-540f8e36b7ba None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:44.701430 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:44.711473 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:44.862144 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d69e5e-2cb6-47a2-a362-c1a9f42df310 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:44.875150 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6743293c-14cf-42a7-8097-540f8e36b7ba None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:44.915367 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9311b643-e0d2-4e09-a67b-6db227ff2591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:44.916048 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311b643-e0d2-4e09-a67b-6db227ff2591 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:44.916419 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311b643-e0d2-4e09-a67b-6db227ff2591 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:44.916695 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311b643-e0d2-4e09-a67b-6db227ff2591 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:44.925548 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8df1fd-c519-4bf1-814a-c227d39f2084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:44.926189 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8df1fd-c519-4bf1-814a-c227d39f2084 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:44.926465 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8df1fd-c519-4bf1-814a-c227d39f2084 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:44.926733 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8df1fd-c519-4bf1-814a-c227d39f2084 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:44.928467 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9311b643-e0d2-4e09-a67b-6db227ff2591 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:44.929939 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9311b643-e0d2-4e09-a67b-6db227ff2591 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:44.938192 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8df1fd-c519-4bf1-814a-c227d39f2084 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:44.939867 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8df1fd-c519-4bf1-814a-c227d39f2084 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:44.946963 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-9311b643-e0d2-4e09-a67b-6db227ff2591 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:44.946963 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:44.946963 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:44.946963 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:44.946963 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:44.946963 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:44.946963 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:44.946963 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:44.946963 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:44.946963 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:44.946963 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:44.946963 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:44.946963 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:44.946963 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:44.946963 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:44.948670 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:44.948670 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:44.948670 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:44.948670 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:44.948670 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:44.948670 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:44.948670 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:44.948670 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:44.948670 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:44.948670 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:44.948670 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:44.948670 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:44.948670 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:44.948670 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:44.948670 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:44.948670 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:44.948670 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:44.949778 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:44.949778 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:44.949778 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:44.949778 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:44.949778 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 360/704] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:44.949778 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15dbb0-f1f1-4ff5-a65c-a2fd6597f25b None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:44.950194 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15dbb0-f1f1-4ff5-a65c-a2fd6597f25b None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:44.950194 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15dbb0-f1f1-4ff5-a65c-a2fd6597f25b None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:44.954170 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 361/705] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:44.954450 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-fc8df1fd-c519-4bf1-814a-c227d39f2084 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:44.954450 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:44.954450 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:44.954450 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:44.954450 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:44.954450 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:44.954450 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:44.954450 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:44.954450 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:44.954450 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:44.954450 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:44.954450 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:44.954450 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 12:38:44.954450 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 12:38:44.954450 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 12:38:44.956706 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 12:38:44.956706 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:44.956706 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:44.956706 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:44.956706 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:44.956706 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:44.956706 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:44.956706 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:44.956706 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:44.956706 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:44.956706 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:44.956706 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:44.956706 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:44.956706 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:44.956706 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:44.956706 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:44.956706 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:44.959141 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 12:38:44.959141 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 12:38:44.959141 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 12:38:44.959141 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 12:38:44.959141 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 12:38:44.959141 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 12:38:44.959141 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:44.959141 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:44.959141 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 345/706] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:38:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:44.959924 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b15dbb0-f1f1-4ff5-a65c-a2fd6597f25b None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:44.960199 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8df1fd-c519-4bf1-814a-c227d39f2084 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:45.054760 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:45.065539 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:45.093566 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b15dbb0-f1f1-4ff5-a65c-a2fd6597f25b None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:45.097921 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8df1fd-c519-4bf1-814a-c227d39f2084 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:45.141658 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04529dc8-7570-4f0a-9fde-1575e355189d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:45.142612 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04529dc8-7570-4f0a-9fde-1575e355189d None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:45.143040 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04529dc8-7570-4f0a-9fde-1575e355189d None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:45.143290 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25432e24-6440-438e-b571-5ed5311ec829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:45.143624 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04529dc8-7570-4f0a-9fde-1575e355189d None admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:45.144127 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25432e24-6440-438e-b571-5ed5311ec829 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:45.144499 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25432e24-6440-438e-b571-5ed5311ec829 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:45.144862 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25432e24-6440-438e-b571-5ed5311ec829 None admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:45.144974 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04529dc8-7570-4f0a-9fde-1575e355189d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:45.145400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04529dc8-7570-4f0a-9fde-1575e355189d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:45.145797 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25432e24-6440-438e-b571-5ed5311ec829 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:45.146199 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25432e24-6440-438e-b571-5ed5311ec829 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:45.148077 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04529dc8-7570-4f0a-9fde-1575e355189d None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:45.148489 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25432e24-6440-438e-b571-5ed5311ec829 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:45.167560 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 362/707] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:45.170880 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 346/708] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 26 12:38:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:45.175619 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c2ecd-e973-46f7-96b5-d72b250cc46c None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:45.175619 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c2ecd-e973-46f7-96b5-d72b250cc46c None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:45.175619 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c2ecd-e973-46f7-96b5-d72b250cc46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:45.180555 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04529dc8-7570-4f0a-9fde-1575e355189d None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:45.198737 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-f50c2ecd-e973-46f7-96b5-d72b250cc46c None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:45.200093 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-f50c2ecd-e973-46f7-96b5-d72b250cc46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:45.279660 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 363/709] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:38:45.279660 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c2ecd-e973-46f7-96b5-d72b250cc46c None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:45.315799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04529dc8-7570-4f0a-9fde-1575e355189d None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:45.341834 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:45.355271 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ca594-a818-4e58-99bf-b80dbd10db5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:45.355851 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ca594-a818-4e58-99bf-b80dbd10db5d None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:45.356130 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ca594-a818-4e58-99bf-b80dbd10db5d None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:45.356712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ca594-a818-4e58-99bf-b80dbd10db5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:45.357797 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8ca594-a818-4e58-99bf-b80dbd10db5d None admin] RBAC: Adding query filter params (role.id=0e78444e43c744668178971af4087ca0, scope.project.id=644278e7e8934b37a1e7ab20d35764cd, user.id=0f5d6f3c12244c0faf26048513dc49e7) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:45.358103 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8ca594-a818-4e58-99bf-b80dbd10db5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:45.359885 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8ca594-a818-4e58-99bf-b80dbd10db5d None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:45.366329 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:45.379406 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c2ecd-e973-46f7-96b5-d72b250cc46c None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:45.380001 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 347/710] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:38:45 2022] GET /identity/v3/role_assignments?user.id=0f5d6f3c12244c0faf26048513dc49e7&scope.project.id=644278e7e8934b37a1e7ab20d35764cd&role.id=0e78444e43c744668178971af4087ca0 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:45.381358 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bdee87-f0a7-4fab-a753-bb6dcd0088ea None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:45.381646 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bdee87-f0a7-4fab-a753-bb6dcd0088ea None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:45.381925 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bdee87-f0a7-4fab-a753-bb6dcd0088ea None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:45.383263 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 348/711] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:45.383995 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bdee87-f0a7-4fab-a753-bb6dcd0088ea None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:45.425156 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f541cd5-42a4-49f4-b90a-cf60a0617bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:45.425987 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f541cd5-42a4-49f4-b90a-cf60a0617bfc None admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:45.426287 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f541cd5-42a4-49f4-b90a-cf60a0617bfc None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:45.426682 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f541cd5-42a4-49f4-b90a-cf60a0617bfc None admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:45.427407 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f541cd5-42a4-49f4-b90a-cf60a0617bfc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:45.430259 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f541cd5-42a4-49f4-b90a-cf60a0617bfc None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:45.466411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 364/712] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Jul 26 12:38:45 2022] POST /identity/v3/users => generated 254 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:38:45.466411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50fa95-a07a-4f6b-859a-0e2fbc1dda12 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:45.466411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50fa95-a07a-4f6b-859a-0e2fbc1dda12 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:45.466411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50fa95-a07a-4f6b-859a-0e2fbc1dda12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:45.488285 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-7d50fa95-a07a-4f6b-859a-0e2fbc1dda12 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:45.494025 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-7d50fa95-a07a-4f6b-859a-0e2fbc1dda12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:45.580245 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bdee87-f0a7-4fab-a753-bb6dcd0088ea None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:45.591721 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 365/713] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:38:45.621627 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b92a3-0589-444c-a095-fc63f2bf2504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:45.622339 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b92a3-0589-444c-a095-fc63f2bf2504 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:45.622623 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b92a3-0589-444c-a095-fc63f2bf2504 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:45.622895 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b92a3-0589-444c-a095-fc63f2bf2504 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:45.623537 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b92a3-0589-444c-a095-fc63f2bf2504 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:45.625146 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b92a3-0589-444c-a095-fc63f2bf2504 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:45.627018 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d50fa95-a07a-4f6b-859a-0e2fbc1dda12 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:45.639652 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-932b92a3-0589-444c-a095-fc63f2bf2504 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 12:38:45.639652 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:45.639652 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 12:38:45.639652 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 12:38:45.639652 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 12:38:45.639652 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 12:38:45.639652 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 12:38:45.639652 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:38:45.639652 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 12:38:45.639652 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:45.639652 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 12:38:45.639652 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:45.639652 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:45.639652 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:45.639652 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:45.639652 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:45.640666 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:45.640666 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:45.640666 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:45.640666 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:45.640666 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:45.640666 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:45.640666 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:45.640666 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 12:38:45.640666 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 12:38:45.640666 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 12:38:45.640666 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 12:38:45.640666 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:45.640666 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:45.640666 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:45.640666 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:45.640666 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:45.641576 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:45.641576 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:45.641576 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:45.641576 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:45.641576 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:45.641576 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:45.641576 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:45.641576 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:45.641576 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:45.641576 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:45.641576 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:45.641576 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 12:38:45.641576 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:38:45.641576 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 12:38:45.641576 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:45.642342 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 349/714] 10.222.0.20 () {56 vars in 1129 bytes} [Tue Jul 26 12:38:45 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:45.651115 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b92a3-0589-444c-a095-fc63f2bf2504 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:45.719626 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:45.752220 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:45.804388 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d50fa95-a07a-4f6b-859a-0e2fbc1dda12 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:45.808993 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b92a3-0589-444c-a095-fc63f2bf2504 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:45.851046 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae13763-c44d-4db4-83bd-048c63f167d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:45.852217 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae13763-c44d-4db4-83bd-048c63f167d1 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:45.852645 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae13763-c44d-4db4-83bd-048c63f167d1 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:45.853126 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae13763-c44d-4db4-83bd-048c63f167d1 None admin] PATH_INFO: `/v3/services/network` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:45.853859 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae13763-c44d-4db4-83bd-048c63f167d1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:45.855514 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae13763-c44d-4db4-83bd-048c63f167d1 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:45.856711 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d3801c-78fc-40ad-803a-2a9069ce074c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:45.857428 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d3801c-78fc-40ad-803a-2a9069ce074c None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:45.857774 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d3801c-78fc-40ad-803a-2a9069ce074c None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:45.858111 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d3801c-78fc-40ad-803a-2a9069ce074c None admin] PATH_INFO: `/v3/services` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:45.858862 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d3801c-78fc-40ad-803a-2a9069ce074c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:45.859843 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d3801c-78fc-40ad-803a-2a9069ce074c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:45.861286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d3801c-78fc-40ad-803a-2a9069ce074c None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:45.869442 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-0ae13763-c44d-4db4-83bd-048c63f167d1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 12:38:45.869442 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:45.869442 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 12:38:45.869442 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 12:38:45.869442 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 12:38:45.869442 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 12:38:45.869442 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 12:38:45.869442 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:38:45.869442 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 12:38:45.869442 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:45.869442 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 12:38:45.869442 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:45.869442 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:45.869442 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:45.869442 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:45.869442 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:45.869442 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:45.872958 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:45.872958 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:45.872958 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:45.872958 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:45.872958 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:45.872958 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:45.872958 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 12:38:45.872958 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 12:38:45.872958 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 12:38:45.872958 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 12:38:45.872958 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:45.872958 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:45.872958 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:45.872958 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:45.872958 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:45.872958 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:45.874165 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:45.874165 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:45.874165 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:45.874165 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:45.874165 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:45.874165 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:45.874165 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:45.874165 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:45.874165 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:45.874165 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:45.874165 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 12:38:45.874165 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:38:45.874165 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 12:38:45.874165 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:45.874165 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 366/716] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:38:45 2022] GET /identity/v3/services/network => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:38:45.879894 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 350/716] 10.222.0.20 () {56 vars in 1124 bytes} [Tue Jul 26 12:38:45 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:45.880063 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae13763-c44d-4db4-83bd-048c63f167d1 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:45.887604 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d3801c-78fc-40ad-803a-2a9069ce074c None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:46.036452 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae13763-c44d-4db4-83bd-048c63f167d1 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:46.046892 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d3801c-78fc-40ad-803a-2a9069ce074c None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:46.067471 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:46.094497 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:46.100870 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564bf436-7f44-44cb-ac45-f1877350ebc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:46.101499 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564bf436-7f44-44cb-ac45-f1877350ebc9 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:46.101781 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564bf436-7f44-44cb-ac45-f1877350ebc9 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:46.102058 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564bf436-7f44-44cb-ac45-f1877350ebc9 None admin] PATH_INFO: `/v3/services` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:46.102751 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564bf436-7f44-44cb-ac45-f1877350ebc9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:46.103055 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564bf436-7f44-44cb-ac45-f1877350ebc9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:46.104458 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564bf436-7f44-44cb-ac45-f1877350ebc9 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:46.110078 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03837337-8263-4292-8f05-2a81732c8a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:46.110760 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03837337-8263-4292-8f05-2a81732c8a08 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:46.111047 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03837337-8263-4292-8f05-2a81732c8a08 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:46.111320 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03837337-8263-4292-8f05-2a81732c8a08 None admin] PATH_INFO: `/v3/services` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:46.112030 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03837337-8263-4292-8f05-2a81732c8a08 None admin] RBAC: Adding query filter params (name=network) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:46.112335 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03837337-8263-4292-8f05-2a81732c8a08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:46.113695 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03837337-8263-4292-8f05-2a81732c8a08 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:46.122065 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 367/717] 10.222.0.20 () {56 vars in 1124 bytes} [Tue Jul 26 12:38:45 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:46.124168 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 351/718] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Jul 26 12:38:45 2022] GET /identity/v3/services?name=network => generated 124 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:38:46.133947 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564bf436-7f44-44cb-ac45-f1877350ebc9 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:46.311661 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564bf436-7f44-44cb-ac45-f1877350ebc9 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:46.353900 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e4040-f5f0-424f-bff9-740470de6f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:46.354540 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e4040-f5f0-424f-bff9-740470de6f42 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:46.354817 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e4040-f5f0-424f-bff9-740470de6f42 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:46.355086 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e4040-f5f0-424f-bff9-740470de6f42 None admin] PATH_INFO: `/v3/services` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:46.355789 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30e4040-f5f0-424f-bff9-740470de6f42 None admin] RBAC: Adding query filter params (type=network) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:46.356092 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30e4040-f5f0-424f-bff9-740470de6f42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:46.357461 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30e4040-f5f0-424f-bff9-740470de6f42 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:46.362379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 368/719] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Jul 26 12:38:46 2022] GET /identity/v3/services?type=network => generated 124 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:47.285290 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21c516-f393-4ff6-a844-6a29d5c31e9f None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:47.285290 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21c516-f393-4ff6-a844-6a29d5c31e9f None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:47.285290 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21c516-f393-4ff6-a844-6a29d5c31e9f None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:47.286390 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 352/720] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:47.292468 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39561a9c-482b-49bd-b4fd-3f97e8065e29 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:47.293383 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39561a9c-482b-49bd-b4fd-3f97e8065e29 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:47.293902 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39561a9c-482b-49bd-b4fd-3f97e8065e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:47.320697 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-39561a9c-482b-49bd-b4fd-3f97e8065e29 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:47.321906 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-39561a9c-482b-49bd-b4fd-3f97e8065e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:47.373846 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:47.414391 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 369/721] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:38:47.450740 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e21c516-f393-4ff6-a844-6a29d5c31e9f None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:47.600693 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e21c516-f393-4ff6-a844-6a29d5c31e9f None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:47.656133 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4783de2-ec56-4d08-adad-be1109ac0a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:47.656833 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4783de2-ec56-4d08-adad-be1109ac0a7c None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:47.657117 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4783de2-ec56-4d08-adad-be1109ac0a7c None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:47.657393 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4783de2-ec56-4d08-adad-be1109ac0a7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:47.676511 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4783de2-ec56-4d08-adad-be1109ac0a7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:38:47.682091 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-a4783de2-ec56-4d08-adad-be1109ac0a7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:38:47.682091 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:47.682091 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:47.682091 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:47.682091 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:47.682091 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:47.682091 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:47.682091 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:47.682091 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:47.682091 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:47.682091 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:47.682091 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:47.682091 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:38:47.682091 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:38:47.682091 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:38:47.682091 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:38:47.683094 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:38:47.683094 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:38:47.683094 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:38:47.683094 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:38:47.683094 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:47.683094 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:47.683094 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:47.683094 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:47.683094 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:47.683094 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:47.683094 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:47.683094 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:47.683094 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:47.683094 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:47.683094 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:47.683094 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:47.683987 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:47.683987 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:47.683987 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:47.683987 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:47.683987 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:38:47.683987 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:38:47.683987 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:38:47.683987 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:38:47.683987 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:38:47.683987 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:38:47.683987 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:38:47.683987 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:47.683987 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 353/722] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 26 12:38:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:47.690964 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39561a9c-482b-49bd-b4fd-3f97e8065e29 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:47.853614 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39561a9c-482b-49bd-b4fd-3f97e8065e29 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:47.906856 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8708a101-3047-4473-b54e-91cc5142dcb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:47.907486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8708a101-3047-4473-b54e-91cc5142dcb6 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:47.907797 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8708a101-3047-4473-b54e-91cc5142dcb6 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:47.908074 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8708a101-3047-4473-b54e-91cc5142dcb6 None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:47.908809 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8708a101-3047-4473-b54e-91cc5142dcb6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:47.909119 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8708a101-3047-4473-b54e-91cc5142dcb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:47.910505 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8708a101-3047-4473-b54e-91cc5142dcb6 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:47.913870 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029af25e-a82a-4d8b-b073-bc72a47912d5 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:47.914163 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029af25e-a82a-4d8b-b073-bc72a47912d5 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:47.914448 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029af25e-a82a-4d8b-b073-bc72a47912d5 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:47.920148 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 370/724] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:38:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:47.920291 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 354/724] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:47.925467 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211041ef-5b29-40bc-aebd-d92c5847c27f None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:47.925758 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211041ef-5b29-40bc-aebd-d92c5847c27f None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:47.926037 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211041ef-5b29-40bc-aebd-d92c5847c27f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:47.926248 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029af25e-a82a-4d8b-b073-bc72a47912d5 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:47.960585 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-211041ef-5b29-40bc-aebd-d92c5847c27f None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:47.962199 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-211041ef-5b29-40bc-aebd-d92c5847c27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:47.996597 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:48.020010 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:48.073837 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 371/725] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:38:48.097293 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029af25e-a82a-4d8b-b073-bc72a47912d5 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:48.098174 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64871fb-298d-41ac-b5a4-494c332d0938 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:48.098459 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64871fb-298d-41ac-b5a4-494c332d0938 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:48.098736 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64871fb-298d-41ac-b5a4-494c332d0938 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:48.102075 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 372/726] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:48.107146 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea3ddf-7fc5-4244-9a8c-60856fcdcd4b None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:48.107363 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea3ddf-7fc5-4244-9a8c-60856fcdcd4b None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:48.107683 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea3ddf-7fc5-4244-9a8c-60856fcdcd4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:48.130016 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-cdea3ddf-7fc5-4244-9a8c-60856fcdcd4b None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:48.131894 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-cdea3ddf-7fc5-4244-9a8c-60856fcdcd4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:48.146041 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20178917-26cf-4c22-9c4b-69442ee96954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:48.147382 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20178917-26cf-4c22-9c4b-69442ee96954 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:48.147830 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20178917-26cf-4c22-9c4b-69442ee96954 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:48.148243 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20178917-26cf-4c22-9c4b-69442ee96954 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:48.161316 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20178917-26cf-4c22-9c4b-69442ee96954 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:48.163924 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20178917-26cf-4c22-9c4b-69442ee96954 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:48.187534 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-20178917-26cf-4c22-9c4b-69442ee96954 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:48.187534 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:48.187534 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:48.187534 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:48.187534 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:48.187534 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:48.187534 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:48.187534 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:48.187534 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:48.187534 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:48.187534 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:48.187534 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:48.187534 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:48.187534 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:48.187534 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:48.188958 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:48.188958 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:48.188958 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:48.188958 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:48.188958 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:48.188958 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:48.188958 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:48.188958 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:48.188958 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:48.188958 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:48.188958 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:48.188958 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:48.188958 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:48.188958 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:48.188958 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:48.188958 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:48.188958 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:48.189853 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:48.189853 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:48.189853 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:48.189853 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:48.192693 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 355/727] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:48.194588 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20178917-26cf-4c22-9c4b-69442ee96954 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:48.231734 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 373/728] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:38:48.232829 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea3ddf-7fc5-4244-9a8c-60856fcdcd4b None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:48.328013 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20178917-26cf-4c22-9c4b-69442ee96954 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:48.331461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:48.358643 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:48.368457 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea3ddf-7fc5-4244-9a8c-60856fcdcd4b None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:48.370900 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de409fc9-7e1d-4e94-a9d9-e19b43cedf6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:48.372396 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de409fc9-7e1d-4e94-a9d9-e19b43cedf6e None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:48.372691 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de409fc9-7e1d-4e94-a9d9-e19b43cedf6e None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:48.372969 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de409fc9-7e1d-4e94-a9d9-e19b43cedf6e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:48.386205 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de409fc9-7e1d-4e94-a9d9-e19b43cedf6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:38:48.392942 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-de409fc9-7e1d-4e94-a9d9-e19b43cedf6e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:38:48.392942 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:48.392942 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:48.392942 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:48.392942 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:48.392942 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:48.392942 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:48.392942 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:48.392942 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:48.392942 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:48.392942 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:48.392942 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:48.392942 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:38:48.392942 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:38:48.392942 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:38:48.392942 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:38:48.393896 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:38:48.393896 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:38:48.393896 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:38:48.393896 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:38:48.393896 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:48.393896 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:48.393896 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:48.393896 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:48.393896 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:48.393896 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:48.393896 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:48.393896 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:48.393896 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:48.393896 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:48.393896 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:48.393896 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:48.394730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:48.394730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:48.394730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:48.394730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:48.394730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:38:48.394730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:38:48.394730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:38:48.394730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:38:48.394730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:38:48.394730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:38:48.394730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:38:48.394730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:48.394730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 356/729] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 26 12:38:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:48.395421 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de409fc9-7e1d-4e94-a9d9-e19b43cedf6e None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:48.416633 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f2b8bf-84ff-433f-ac73-b87bbf8e2a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:48.417314 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2b8bf-84ff-433f-ac73-b87bbf8e2a45 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:48.417601 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2b8bf-84ff-433f-ac73-b87bbf8e2a45 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:48.417880 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2b8bf-84ff-433f-ac73-b87bbf8e2a45 None admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:48.419016 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f2b8bf-84ff-433f-ac73-b87bbf8e2a45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:48.419328 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f2b8bf-84ff-433f-ac73-b87bbf8e2a45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:48.420741 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f2b8bf-84ff-433f-ac73-b87bbf8e2a45 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:48.439681 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 374/730] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:48.439681 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f2b8bf-84ff-433f-ac73-b87bbf8e2a45 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:48.516716 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de409fc9-7e1d-4e94-a9d9-e19b43cedf6e None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:48.553669 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78becd21-854c-440c-aab0-a4940798c375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:48.554325 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78becd21-854c-440c-aab0-a4940798c375 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:48.554587 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78becd21-854c-440c-aab0-a4940798c375 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:48.554860 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78becd21-854c-440c-aab0-a4940798c375 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:48.560780 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f2b8bf-84ff-433f-ac73-b87bbf8e2a45 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:48.565736 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78becd21-854c-440c-aab0-a4940798c375 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:38:48.571258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-78becd21-854c-440c-aab0-a4940798c375 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 12:38:48.571258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:48.571258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:48.571258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:48.571258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:48.571258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:48.571258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:48.571258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:48.571258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:48.571258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:48.571258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:48.571258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:48.571258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:38:48.571258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:38:48.571258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:38:48.571258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:38:48.572257 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:38:48.572257 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:38:48.572257 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:38:48.572257 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:38:48.572257 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:48.572257 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:48.572257 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:48.572257 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:48.572257 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:48.572257 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:48.572257 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:48.572257 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:48.572257 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:48.572257 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:48.572257 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:48.572257 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:48.573088 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:48.573088 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:48.573088 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:48.573088 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:48.573088 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:38:48.573088 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:38:48.573088 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:38:48.573088 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:38:48.573088 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:38:48.573088 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:38:48.573088 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 12:38:48.573088 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:48.573088 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 357/731] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 26 12:38:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:48.573766 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78becd21-854c-440c-aab0-a4940798c375 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:48.602434 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:48.603291 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c37d77-7f09-46ca-92a1-e79fcbb5a4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:48.604646 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c37d77-7f09-46ca-92a1-e79fcbb5a4ba None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:48.604932 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c37d77-7f09-46ca-92a1-e79fcbb5a4ba None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:48.605210 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c37d77-7f09-46ca-92a1-e79fcbb5a4ba None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:48.605921 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c37d77-7f09-46ca-92a1-e79fcbb5a4ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:48.606228 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c37d77-7f09-46ca-92a1-e79fcbb5a4ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:48.607595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c37d77-7f09-46ca-92a1-e79fcbb5a4ba None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:48.617591 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 375/732] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:38:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:48.618633 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c54b6-e18f-465b-b304-cdb62d7ca077 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:48.618921 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c54b6-e18f-465b-b304-cdb62d7ca077 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:48.619195 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c54b6-e18f-465b-b304-cdb62d7ca077 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:48.624245 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 376/733] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:48.624245 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6c54b6-e18f-465b-b304-cdb62d7ca077 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:48.644410 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:48.690373 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78becd21-854c-440c-aab0-a4940798c375 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:48.727908 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab39376-f712-4c82-ab30-7176f029ff8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:48.728556 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab39376-f712-4c82-ab30-7176f029ff8e None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:48.728839 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab39376-f712-4c82-ab30-7176f029ff8e None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:48.729113 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab39376-f712-4c82-ab30-7176f029ff8e None admin] PATH_INFO: `/v3/users/swift` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:48.736683 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6c54b6-e18f-465b-b304-cdb62d7ca077 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:48.737962 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab39376-f712-4c82-ab30-7176f029ff8e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:48.739414 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab39376-f712-4c82-ab30-7176f029ff8e None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:48.751914 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-cab39376-f712-4c82-ab30-7176f029ff8e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 12:38:48.751914 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:48.751914 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:48.751914 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:48.751914 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:48.751914 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:48.751914 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:48.751914 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:48.751914 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:48.751914 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:48.751914 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:48.751914 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:48.751914 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 12:38:48.751914 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 12:38:48.751914 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 12:38:48.751914 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 12:38:48.752868 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:48.752868 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:48.752868 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 12:38:48.752868 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:48.752868 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 12:38:48.752868 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:48.752868 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:48.752868 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:48.752868 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:48.752868 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:48.752868 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:48.752868 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:48.752868 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:48.752868 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:48.752868 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:48.752868 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:48.752868 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:48.752868 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:48.753823 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:48.753823 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:48.753823 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 12:38:48.753823 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 12:38:48.753823 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 12:38:48.753823 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 12:38:48.753823 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 12:38:48.753823 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 12:38:48.753823 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 12:38:48.753823 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:48.753823 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 358/734] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 26 12:38:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:48.753823 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab39376-f712-4c82-ab30-7176f029ff8e None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:48.780558 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3675ec-73b3-49b6-b60c-db36185c6373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:48.781440 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3675ec-73b3-49b6-b60c-db36185c6373 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:48.781876 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3675ec-73b3-49b6-b60c-db36185c6373 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:48.782289 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3675ec-73b3-49b6-b60c-db36185c6373 None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:48.783146 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3675ec-73b3-49b6-b60c-db36185c6373 None admin] RBAC: Adding query filter params (name=service) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:48.783592 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3675ec-73b3-49b6-b60c-db36185c6373 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:48.785150 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3675ec-73b3-49b6-b60c-db36185c6373 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:48.796003 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 377/735] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Jul 26 12:38:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:48.797378 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bdef20-8a5d-4b45-a353-634a6aef17f9 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:48.797817 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bdef20-8a5d-4b45-a353-634a6aef17f9 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:48.798228 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bdef20-8a5d-4b45-a353-634a6aef17f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:48.816648 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-e9bdef20-8a5d-4b45-a353-634a6aef17f9 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:48.817832 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-e9bdef20-8a5d-4b45-a353-634a6aef17f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:48.849264 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:48.868220 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab39376-f712-4c82-ab30-7176f029ff8e None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:48.889027 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 378/736] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:38:48.890242 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bdef20-8a5d-4b45-a353-634a6aef17f9 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:48.903581 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:48.926347 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706a453c-4ec0-429d-b202-3b51279f289f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:48.927289 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a453c-4ec0-429d-b202-3b51279f289f None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:48.927790 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a453c-4ec0-429d-b202-3b51279f289f None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:48.928270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a453c-4ec0-429d-b202-3b51279f289f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:48.940549 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706a453c-4ec0-429d-b202-3b51279f289f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:48.942390 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706a453c-4ec0-429d-b202-3b51279f289f None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:48.959476 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-706a453c-4ec0-429d-b202-3b51279f289f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:48.959476 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:48.959476 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:48.959476 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:48.959476 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:48.959476 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:48.959476 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:48.959476 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:48.959476 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:48.959476 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:48.959476 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:48.959476 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:48.959476 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:48.959476 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:48.959476 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:48.960428 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:48.960428 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:48.960428 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:48.960428 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:48.960428 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:48.960428 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:48.960428 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:48.960428 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:48.960428 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:48.960428 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:48.960428 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:48.960428 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:48.960428 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:48.960428 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:48.960428 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:48.960428 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:48.960428 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:48.966397 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:48.966397 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:48.966397 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:48.966397 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:48.966397 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 359/737] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:48.966397 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706a453c-4ec0-429d-b202-3b51279f289f None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:49.001188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bdef20-8a5d-4b45-a353-634a6aef17f9 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:49.035407 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545dc78f-8d07-47df-b49c-7f3180bb329a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:49.036043 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545dc78f-8d07-47df-b49c-7f3180bb329a None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:49.036332 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545dc78f-8d07-47df-b49c-7f3180bb329a None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:49.036612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545dc78f-8d07-47df-b49c-7f3180bb329a None admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:49.037457 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545dc78f-8d07-47df-b49c-7f3180bb329a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:49.037766 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545dc78f-8d07-47df-b49c-7f3180bb329a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:49.039199 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545dc78f-8d07-47df-b49c-7f3180bb329a None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:49.056775 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 379/738] 10.222.0.20 () {56 vars in 1135 bytes} [Tue Jul 26 12:38:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:49.057744 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545dc78f-8d07-47df-b49c-7f3180bb329a None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:49.061413 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706a453c-4ec0-429d-b202-3b51279f289f None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:49.095275 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcfbb06-a4ad-4cc2-ab40-e1ef55f01985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:49.095963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcfbb06-a4ad-4cc2-ab40-e1ef55f01985 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:49.096245 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcfbb06-a4ad-4cc2-ab40-e1ef55f01985 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:49.096524 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcfbb06-a4ad-4cc2-ab40-e1ef55f01985 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:49.099181 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:49.108025 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcfbb06-a4ad-4cc2-ab40-e1ef55f01985 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:49.109544 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcfbb06-a4ad-4cc2-ab40-e1ef55f01985 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:49.125400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-9fcfbb06-a4ad-4cc2-ab40-e1ef55f01985 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:49.125400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:49.125400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:49.125400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:49.125400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:49.125400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:49.125400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:49.125400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:49.125400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:49.125400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:49.125400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:49.125400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:49.125400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:49.125400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:49.125400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:49.127045 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:49.127045 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:49.127045 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:49.127045 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:49.127045 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:49.127045 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:49.127045 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:49.127045 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:49.127045 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:49.127045 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:49.127045 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:49.127045 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:49.127045 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:49.127045 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:49.127045 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:49.127045 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:49.127045 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:49.129915 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:49.129915 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:49.129915 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:49.129915 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:49.130439 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:49.130566 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 360/739] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:49.130566 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcfbb06-a4ad-4cc2-ab40-e1ef55f01985 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:49.165144 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545dc78f-8d07-47df-b49c-7f3180bb329a None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:49.202622 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23666791-485d-40be-a4b4-af0d54e3244e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:49.203230 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23666791-485d-40be-a4b4-af0d54e3244e None admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:49.203506 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23666791-485d-40be-a4b4-af0d54e3244e None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:49.203813 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23666791-485d-40be-a4b4-af0d54e3244e None admin] PATH_INFO: `/v3/services` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:49.204498 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23666791-485d-40be-a4b4-af0d54e3244e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:49.205865 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23666791-485d-40be-a4b4-af0d54e3244e None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:49.222672 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 380/740] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Jul 26 12:38:49 2022] POST /identity/v3/services => generated 251 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:38:49.223902 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5b455-8f05-4103-9250-a7e25e81068c None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:49.224188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5b455-8f05-4103-9250-a7e25e81068c None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:49.224459 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5b455-8f05-4103-9250-a7e25e81068c None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:49.227704 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 381/741] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:49.227704 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc5b455-8f05-4103-9250-a7e25e81068c None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:49.246426 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcfbb06-a4ad-4cc2-ab40-e1ef55f01985 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:49.300626 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff275e73-defe-459b-8b37-aa55bf545b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:49.301684 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff275e73-defe-459b-8b37-aa55bf545b5e None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:49.302121 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff275e73-defe-459b-8b37-aa55bf545b5e None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:49.303112 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff275e73-defe-459b-8b37-aa55bf545b5e None admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:49.304151 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff275e73-defe-459b-8b37-aa55bf545b5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:49.304803 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff275e73-defe-459b-8b37-aa55bf545b5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:49.306562 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff275e73-defe-459b-8b37-aa55bf545b5e None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:49.325444 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 361/742] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:49.325444 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff275e73-defe-459b-8b37-aa55bf545b5e None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:49.352911 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc5b455-8f05-4103-9250-a7e25e81068c None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:49.373864 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:49.387851 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:49.400488 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f288a70-a7b0-4b74-b350-2ce2a005a477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:49.401950 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f288a70-a7b0-4b74-b350-2ce2a005a477 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:49.402239 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f288a70-a7b0-4b74-b350-2ce2a005a477 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:49.402517 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f288a70-a7b0-4b74-b350-2ce2a005a477 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:49.437534 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff275e73-defe-459b-8b37-aa55bf545b5e None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:49.471860 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53556fc-e518-4c7a-8d19-e761e07bd059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:49.472483 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53556fc-e518-4c7a-8d19-e761e07bd059 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:49.472765 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53556fc-e518-4c7a-8d19-e761e07bd059 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:49.473036 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53556fc-e518-4c7a-8d19-e761e07bd059 None admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:49.473727 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53556fc-e518-4c7a-8d19-e761e07bd059 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:49.474033 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53556fc-e518-4c7a-8d19-e761e07bd059 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:49.476007 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53556fc-e518-4c7a-8d19-e761e07bd059 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:49.476703 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f288a70-a7b0-4b74-b350-2ce2a005a477 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:49.478346 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f288a70-a7b0-4b74-b350-2ce2a005a477 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:49.488798 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 362/743] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:49.490654 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745518a-6f17-48d6-bd83-79826c1ff34c None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:49.490938 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745518a-6f17-48d6-bd83-79826c1ff34c None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:49.491209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745518a-6f17-48d6-bd83-79826c1ff34c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:49.604649 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 382/744] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:38:49 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 12:38:49.604649 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f288a70-a7b0-4b74-b350-2ce2a005a477 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:49.626237 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-e745518a-6f17-48d6-bd83-79826c1ff34c None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:49.627535 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-e745518a-6f17-48d6-bd83-79826c1ff34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:49.634609 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:49.706627 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 363/745] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:49 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 12:38:49.707496 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e745518a-6f17-48d6-bd83-79826c1ff34c None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:49.723985 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f288a70-a7b0-4b74-b350-2ce2a005a477 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:49.742536 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:49.766195 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945bfcb3-edc3-4c00-a216-396fb172e2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:49.766904 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945bfcb3-edc3-4c00-a216-396fb172e2d1 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:49.767183 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945bfcb3-edc3-4c00-a216-396fb172e2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:49.767556 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945bfcb3-edc3-4c00-a216-396fb172e2d1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:49.776372 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945bfcb3-edc3-4c00-a216-396fb172e2d1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:49.777860 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945bfcb3-edc3-4c00-a216-396fb172e2d1 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:49.791930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-945bfcb3-edc3-4c00-a216-396fb172e2d1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 12:38:49.791930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:49.791930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:49.791930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:49.791930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:49.791930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:49.791930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:49.791930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:49.791930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:49.791930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:49.791930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:49.791930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:49.791930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 12:38:49.791930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 12:38:49.791930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 12:38:49.791930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 12:38:49.793183 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:49.793183 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:49.793183 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 12:38:49.793183 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:49.793183 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 12:38:49.793183 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:49.793183 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:49.793183 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:49.793183 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:49.793183 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:49.793183 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:49.793183 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:49.793183 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:49.793183 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:49.793183 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:49.793183 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:49.793183 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:49.793183 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:49.794181 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:49.794181 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:49.794181 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 12:38:49.794181 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 12:38:49.794181 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 12:38:49.794181 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 12:38:49.794181 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 12:38:49.794181 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 12:38:49.794181 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 12:38:49.794181 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:49.794181 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 383/746] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Jul 26 12:38:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:49.794181 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945bfcb3-edc3-4c00-a216-396fb172e2d1 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:49.820504 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e745518a-6f17-48d6-bd83-79826c1ff34c None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:49.858196 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f847b3a8-e66a-4ce5-a23e-ddd64d4ee3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:49.858956 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f847b3a8-e66a-4ce5-a23e-ddd64d4ee3f8 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:49.859242 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f847b3a8-e66a-4ce5-a23e-ddd64d4ee3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:49.859514 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f847b3a8-e66a-4ce5-a23e-ddd64d4ee3f8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:49.867848 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f847b3a8-e66a-4ce5-a23e-ddd64d4ee3f8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:49.869312 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f847b3a8-e66a-4ce5-a23e-ddd64d4ee3f8 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:49.882812 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-f847b3a8-e66a-4ce5-a23e-ddd64d4ee3f8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 12:38:49.882812 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:49.882812 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:49.882812 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:49.882812 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:49.882812 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:49.882812 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:49.882812 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:49.882812 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:49.882812 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:49.882812 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:49.882812 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:49.882812 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 12:38:49.882812 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 12:38:49.882812 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 12:38:49.882812 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 12:38:49.883918 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:49.883918 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:49.883918 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 12:38:49.883918 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:49.883918 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 12:38:49.883918 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:49.883918 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:49.883918 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:49.883918 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:49.883918 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:49.883918 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:49.883918 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:49.883918 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:49.883918 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:49.883918 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:49.883918 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:49.883918 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:49.883918 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:49.884855 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:49.884855 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:49.884855 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 12:38:49.884855 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 12:38:49.884855 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 12:38:49.884855 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 12:38:49.884855 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 12:38:49.884855 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 12:38:49.884855 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 12:38:49.884855 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:49.884855 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 364/747] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Jul 26 12:38:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:49.884855 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f847b3a8-e66a-4ce5-a23e-ddd64d4ee3f8 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:49.910158 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:49.922817 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945bfcb3-edc3-4c00-a216-396fb172e2d1 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:49.959969 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d100e0de-41ee-406d-b8e5-c8636f58c34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:49.960678 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d100e0de-41ee-406d-b8e5-c8636f58c34f None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:49.961000 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d100e0de-41ee-406d-b8e5-c8636f58c34f None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:49.961285 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d100e0de-41ee-406d-b8e5-c8636f58c34f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:49.972618 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d100e0de-41ee-406d-b8e5-c8636f58c34f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:49.974084 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d100e0de-41ee-406d-b8e5-c8636f58c34f None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:49.989554 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-d100e0de-41ee-406d-b8e5-c8636f58c34f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:49.989554 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:49.989554 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:49.989554 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:49.989554 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:49.989554 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:49.989554 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:49.989554 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:49.989554 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:49.989554 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:49.989554 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:49.989554 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:49.989554 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:49.989554 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:49.989554 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:49.991422 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:49.991422 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:49.991422 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:49.991422 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:49.991422 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:49.991422 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:49.991422 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:49.991422 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:49.991422 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:49.991422 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:49.991422 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:49.991422 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:49.991422 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:49.991422 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:49.991422 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:49.991422 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:49.991422 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:49.992372 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:49.992372 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:49.992372 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:49.992372 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:49.992372 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 384/748] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:49.992372 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d100e0de-41ee-406d-b8e5-c8636f58c34f None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:49.995055 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:50.003829 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f847b3a8-e66a-4ce5-a23e-ddd64d4ee3f8 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:50.039114 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99709da4-da26-4181-bdb7-0a5f0ddd0503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:50.039873 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99709da4-da26-4181-bdb7-0a5f0ddd0503 None admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:50.040217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99709da4-da26-4181-bdb7-0a5f0ddd0503 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:50.040565 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99709da4-da26-4181-bdb7-0a5f0ddd0503 None admin] PATH_INFO: `/v3/services` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:50.041314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99709da4-da26-4181-bdb7-0a5f0ddd0503 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:50.042675 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99709da4-da26-4181-bdb7-0a5f0ddd0503 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:50.059677 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 365/749] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Jul 26 12:38:49 2022] POST /identity/v3/services => generated 240 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:38:50.059677 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99709da4-da26-4181-bdb7-0a5f0ddd0503 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:50.137281 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d100e0de-41ee-406d-b8e5-c8636f58c34f None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:50.202751 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5d9ac-ac74-4f6a-899e-94572c29c25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:50.203548 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5d9ac-ac74-4f6a-899e-94572c29c25c None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:50.203961 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5d9ac-ac74-4f6a-899e-94572c29c25c None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:50.204320 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5d9ac-ac74-4f6a-899e-94572c29c25c None admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:50.205123 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99709da4-da26-4181-bdb7-0a5f0ddd0503 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:50.205307 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5d9ac-ac74-4f6a-899e-94572c29c25c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:50.205711 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5d9ac-ac74-4f6a-899e-94572c29c25c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:50.207287 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5d9ac-ac74-4f6a-899e-94572c29c25c None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:50.212690 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:50.236018 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 385/750] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Jul 26 12:38:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:50.237067 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5d9ac-ac74-4f6a-899e-94572c29c25c None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:50.255427 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199ce836-7d82-4fb4-ba01-24ff03f66f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:50.256647 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ce836-7d82-4fb4-ba01-24ff03f66f62 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:50.256945 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ce836-7d82-4fb4-ba01-24ff03f66f62 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:50.257196 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ce836-7d82-4fb4-ba01-24ff03f66f62 None admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:50.258133 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199ce836-7d82-4fb4-ba01-24ff03f66f62 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:50.258331 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199ce836-7d82-4fb4-ba01-24ff03f66f62 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:50.260502 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199ce836-7d82-4fb4-ba01-24ff03f66f62 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:50.285701 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 366/751] 10.222.0.20 () {56 vars in 1149 bytes} [Tue Jul 26 12:38:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:50.286644 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199ce836-7d82-4fb4-ba01-24ff03f66f62 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:50.299093 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:50.374627 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5d9ac-ac74-4f6a-899e-94572c29c25c None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:50.401948 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199ce836-7d82-4fb4-ba01-24ff03f66f62 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:50.425828 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed57a1d2-0ea8-43c7-9caa-f0ffeb089d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:50.427881 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57a1d2-0ea8-43c7-9caa-f0ffeb089d2b None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:50.428511 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57a1d2-0ea8-43c7-9caa-f0ffeb089d2b None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:50.429094 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57a1d2-0ea8-43c7-9caa-f0ffeb089d2b None admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:50.430267 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed57a1d2-0ea8-43c7-9caa-f0ffeb089d2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:50.431167 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed57a1d2-0ea8-43c7-9caa-f0ffeb089d2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:50.433321 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed57a1d2-0ea8-43c7-9caa-f0ffeb089d2b None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:50.438748 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bcf62-4311-47b9-adae-e029a0948ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:50.439389 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bcf62-4311-47b9-adae-e029a0948ef7 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:50.439700 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bcf62-4311-47b9-adae-e029a0948ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:50.439976 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bcf62-4311-47b9-adae-e029a0948ef7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:50.452313 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 386/752] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:50.453363 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310bcf62-4311-47b9-adae-e029a0948ef7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:50.453750 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed57a1d2-0ea8-43c7-9caa-f0ffeb089d2b None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:50.455052 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310bcf62-4311-47b9-adae-e029a0948ef7 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:50.471021 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-310bcf62-4311-47b9-adae-e029a0948ef7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:50.471021 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:50.471021 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:50.471021 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:50.471021 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:50.471021 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:50.471021 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:50.471021 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:50.471021 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:50.471021 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:50.471021 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:50.471021 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:50.471021 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:50.471021 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:50.471021 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:50.472177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:50.472177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:50.472177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:50.472177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:50.472177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:50.472177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:50.472177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:50.472177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:50.472177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:50.472177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:50.472177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:50.472177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:50.472177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:50.472177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:50.472177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:50.472177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:50.472177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:50.473278 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:50.473278 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:50.473278 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:50.473278 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:50.475320 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 367/753] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:50.476235 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bcf62-4311-47b9-adae-e029a0948ef7 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:50.542392 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:50.551824 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:50.601302 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bcf62-4311-47b9-adae-e029a0948ef7 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:50.606628 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed57a1d2-0ea8-43c7-9caa-f0ffeb089d2b None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:50.644096 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0740c9c6-2965-48b3-a801-99996b3275fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:50.644658 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731878cd-8975-48b8-8373-d83ce07c7a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:50.645303 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731878cd-8975-48b8-8373-d83ce07c7a76 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:50.645548 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740c9c6-2965-48b3-a801-99996b3275fb None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:50.645652 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731878cd-8975-48b8-8373-d83ce07c7a76 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:50.645852 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731878cd-8975-48b8-8373-d83ce07c7a76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:50.646334 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740c9c6-2965-48b3-a801-99996b3275fb None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:50.646818 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740c9c6-2965-48b3-a801-99996b3275fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:50.654104 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731878cd-8975-48b8-8373-d83ce07c7a76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:50.655560 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731878cd-8975-48b8-8373-d83ce07c7a76 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:50.671607 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-731878cd-8975-48b8-8373-d83ce07c7a76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:50.671607 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:50.671607 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:50.671607 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:50.671607 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:50.671607 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:50.671607 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:50.671607 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:50.671607 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:50.671607 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:50.671607 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:50.671607 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:50.671607 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 12:38:50.671607 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 12:38:50.671607 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 12:38:50.672604 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 12:38:50.672604 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:50.672604 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:50.672604 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:50.672604 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:50.672604 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:50.672604 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:50.672604 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:50.672604 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:50.672604 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:50.672604 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:50.672604 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:50.672604 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:50.672604 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:50.672604 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:50.672604 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:50.672604 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:50.674095 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 12:38:50.674095 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 12:38:50.674095 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 12:38:50.674095 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 12:38:50.674095 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 12:38:50.674095 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 12:38:50.674095 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:50.674095 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:50.674095 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 368/754] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:38:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:50.674095 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731878cd-8975-48b8-8373-d83ce07c7a76 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:50.734236 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0740c9c6-2965-48b3-a801-99996b3275fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:50.736058 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0740c9c6-2965-48b3-a801-99996b3275fb None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:50.798978 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731878cd-8975-48b8-8373-d83ce07c7a76 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:50.799331 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:50.824221 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:50.843865 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e011d-861d-4fb5-8689-4e0825d4ec84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:50.844975 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e011d-861d-4fb5-8689-4e0825d4ec84 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:50.845458 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e011d-861d-4fb5-8689-4e0825d4ec84 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:50.845929 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e011d-861d-4fb5-8689-4e0825d4ec84 None admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:50.846819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e011d-861d-4fb5-8689-4e0825d4ec84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:50.847324 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e011d-861d-4fb5-8689-4e0825d4ec84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:50.849226 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e011d-861d-4fb5-8689-4e0825d4ec84 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:50.864482 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 387/755] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:38:50 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 12:38:50.864482 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0740c9c6-2965-48b3-a801-99996b3275fb None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:50.868826 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 369/756] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:50.870048 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e011d-861d-4fb5-8689-4e0825d4ec84 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:50.985505 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0740c9c6-2965-48b3-a801-99996b3275fb None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:50.999258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e011d-861d-4fb5-8689-4e0825d4ec84 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:51.031617 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ed035-7ce2-4dbe-8fa6-e41e4ea4adcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:51.032533 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ed035-7ce2-4dbe-8fa6-e41e4ea4adcc None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:51.032819 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ed035-7ce2-4dbe-8fa6-e41e4ea4adcc None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:51.033099 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ed035-7ce2-4dbe-8fa6-e41e4ea4adcc None admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:51.033857 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522ed035-7ce2-4dbe-8fa6-e41e4ea4adcc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:51.034177 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522ed035-7ce2-4dbe-8fa6-e41e4ea4adcc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:51.035618 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522ed035-7ce2-4dbe-8fa6-e41e4ea4adcc None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:51.038240 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689dcc6f-35f8-4e11-a92b-a3e0bf4ceb3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:51.039231 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689dcc6f-35f8-4e11-a92b-a3e0bf4ceb3c None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:51.039785 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689dcc6f-35f8-4e11-a92b-a3e0bf4ceb3c None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:51.040273 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689dcc6f-35f8-4e11-a92b-a3e0bf4ceb3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:51.049268 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 388/757] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 26 12:38:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:51.051862 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ed035-7ce2-4dbe-8fa6-e41e4ea4adcc None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:51.058676 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689dcc6f-35f8-4e11-a92b-a3e0bf4ceb3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:51.060205 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689dcc6f-35f8-4e11-a92b-a3e0bf4ceb3c None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:51.070873 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:51.080638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-689dcc6f-35f8-4e11-a92b-a3e0bf4ceb3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:51.080638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:51.080638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:51.080638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:51.080638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:51.080638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:51.080638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:51.080638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:51.080638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:51.080638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:51.080638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:51.080638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:51.080638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:51.080638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:51.080638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:51.081536 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:51.081536 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:51.081536 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:51.081536 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:51.081536 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:51.081536 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:51.081536 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:51.081536 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:51.081536 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:51.081536 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:51.081536 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:51.081536 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:51.081536 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:51.081536 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:51.081536 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:51.081536 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:51.081536 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:51.082827 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:51.082827 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:51.082827 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:51.082827 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:51.082827 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 370/758] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:51.082827 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689dcc6f-35f8-4e11-a92b-a3e0bf4ceb3c None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:51.125485 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:51.181156 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ed035-7ce2-4dbe-8fa6-e41e4ea4adcc None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:51.204133 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689dcc6f-35f8-4e11-a92b-a3e0bf4ceb3c None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:51.227167 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b21af-08bc-4379-8320-a99325376717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:51.235835 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b21af-08bc-4379-8320-a99325376717 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:51.235835 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b21af-08bc-4379-8320-a99325376717 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:51.235835 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b21af-08bc-4379-8320-a99325376717 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:51.245684 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eb6c46-3102-4fd0-9a4e-c0de15a7b5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:51.246439 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eb6c46-3102-4fd0-9a4e-c0de15a7b5b0 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:51.246826 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eb6c46-3102-4fd0-9a4e-c0de15a7b5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:51.247201 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eb6c46-3102-4fd0-9a4e-c0de15a7b5b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:51.247465 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7b21af-08bc-4379-8320-a99325376717 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:51.248234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eb6c46-3102-4fd0-9a4e-c0de15a7b5b0 None admin] RBAC: Adding query filter params (role.id=0e78444e43c744668178971af4087ca0, scope.project.id=644278e7e8934b37a1e7ab20d35764cd, user.id=cd2483f74d7d48b1a7b8ffe32f63abde) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:51.248649 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eb6c46-3102-4fd0-9a4e-c0de15a7b5b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:51.249454 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7b21af-08bc-4379-8320-a99325376717 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:51.250312 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eb6c46-3102-4fd0-9a4e-c0de15a7b5b0 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:51.263667 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 371/759] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:38:51 2022] GET /identity/v3/role_assignments?user.id=cd2483f74d7d48b1a7b8ffe32f63abde&scope.project.id=644278e7e8934b37a1e7ab20d35764cd&role.id=0e78444e43c744668178971af4087ca0 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:51.263667 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eb6c46-3102-4fd0-9a4e-c0de15a7b5b0 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:51.279356 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-1e7b21af-08bc-4379-8320-a99325376717 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:51.279356 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:51.279356 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:51.279356 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:51.279356 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:51.279356 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:51.279356 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:51.279356 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:51.279356 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:51.279356 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:51.279356 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:51.279356 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:51.279356 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 12:38:51.279356 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 12:38:51.279356 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 12:38:51.280376 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 12:38:51.280376 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:51.280376 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:51.280376 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:51.280376 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:51.280376 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:51.280376 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:51.280376 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:51.280376 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:51.280376 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:51.280376 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:51.280376 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:51.280376 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:51.280376 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:51.280376 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:51.280376 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:51.280376 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:51.281267 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 12:38:51.281267 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 12:38:51.281267 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 12:38:51.281267 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 12:38:51.281267 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 12:38:51.281267 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 12:38:51.281267 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:51.281267 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:51.289648 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 389/760] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:38:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:51.290396 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b21af-08bc-4379-8320-a99325376717 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:51.394414 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:51.401907 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eb6c46-3102-4fd0-9a4e-c0de15a7b5b0 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:51.405822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:51.442759 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee46facf-7c0a-47f6-beb9-985751f7cc32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:51.443472 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46facf-7c0a-47f6-beb9-985751f7cc32 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:51.443790 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46facf-7c0a-47f6-beb9-985751f7cc32 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:51.444061 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46facf-7c0a-47f6-beb9-985751f7cc32 None admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:51.444803 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee46facf-7c0a-47f6-beb9-985751f7cc32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:51.445106 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee46facf-7c0a-47f6-beb9-985751f7cc32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:51.446459 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee46facf-7c0a-47f6-beb9-985751f7cc32 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:51.448729 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b21af-08bc-4379-8320-a99325376717 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:51.460749 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 372/761] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:51.462026 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1dacec-b6ca-45d4-897a-c7dfedd90245 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:51.462310 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1dacec-b6ca-45d4-897a-c7dfedd90245 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:51.462585 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1dacec-b6ca-45d4-897a-c7dfedd90245 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:51.464193 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 373/762] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:51.469667 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421cff3-2b96-4de4-84f6-f6fc83c14955 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:51.470086 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421cff3-2b96-4de4-84f6-f6fc83c14955 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:51.470462 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421cff3-2b96-4de4-84f6-f6fc83c14955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:51.492236 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92768137-03fe-41ac-933b-70e2e4300e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:51.493126 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92768137-03fe-41ac-933b-70e2e4300e70 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:51.493533 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-f421cff3-2b96-4de4-84f6-f6fc83c14955 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:51.493707 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92768137-03fe-41ac-933b-70e2e4300e70 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:51.494140 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92768137-03fe-41ac-933b-70e2e4300e70 None admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:51.495035 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-f421cff3-2b96-4de4-84f6-f6fc83c14955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:51.495204 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92768137-03fe-41ac-933b-70e2e4300e70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:51.495664 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92768137-03fe-41ac-933b-70e2e4300e70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:51.497244 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92768137-03fe-41ac-933b-70e2e4300e70 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:51.516224 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 390/763] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 26 12:38:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:51.517424 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92768137-03fe-41ac-933b-70e2e4300e70 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:51.581813 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 374/764] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:38:51.583005 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f421cff3-2b96-4de4-84f6-f6fc83c14955 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:51.655509 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92768137-03fe-41ac-933b-70e2e4300e70 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:51.672021 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:51.689958 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f421cff3-2b96-4de4-84f6-f6fc83c14955 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:51.692218 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ffcdd-81e4-4016-b415-4d5df06ba972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:51.693530 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ffcdd-81e4-4016-b415-4d5df06ba972 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:51.693530 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ffcdd-81e4-4016-b415-4d5df06ba972 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:51.693530 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ffcdd-81e4-4016-b415-4d5df06ba972 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:51.695458 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:51.702163 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13ffcdd-81e4-4016-b415-4d5df06ba972 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:51.703734 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13ffcdd-81e4-4016-b415-4d5df06ba972 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:51.715907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-e13ffcdd-81e4-4016-b415-4d5df06ba972 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:51.715907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:51.715907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:51.715907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:51.715907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:51.715907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:51.715907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:51.715907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:51.715907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:51.715907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:51.715907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:51.715907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:51.715907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 12:38:51.715907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 12:38:51.715907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 12:38:51.716878 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 12:38:51.716878 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:51.716878 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:51.716878 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:51.716878 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:51.716878 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:51.716878 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:51.716878 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:51.716878 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:51.716878 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:51.716878 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:51.716878 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:51.716878 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:51.716878 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:51.716878 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:51.716878 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:51.716878 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:51.717766 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 12:38:51.717766 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 12:38:51.717766 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 12:38:51.717766 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 12:38:51.717766 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 12:38:51.717766 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 12:38:51.717766 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:51.717766 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:51.721974 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 391/765] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:38:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:51.722924 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ffcdd-81e4-4016-b415-4d5df06ba972 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:51.732850 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a47eae-2270-46ea-8c6a-0b520849a436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:51.733762 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a47eae-2270-46ea-8c6a-0b520849a436 None admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:51.734181 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a47eae-2270-46ea-8c6a-0b520849a436 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:51.734593 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a47eae-2270-46ea-8c6a-0b520849a436 None admin] PATH_INFO: `/v3/projects/644278e7e8934b37a1e7ab20d35764cd/users/0f5d6f3c12244c0faf26048513dc49e7/roles/0e78444e43c744668178971af4087ca0` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:51.736760 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a47eae-2270-46ea-8c6a-0b520849a436 None admin] RBAC: Authorizing `identity:create_grant(project_id=644278e7e8934b37a1e7ab20d35764cd, user_id=0f5d6f3c12244c0faf26048513dc49e7, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:51.738377 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a47eae-2270-46ea-8c6a-0b520849a436 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:51.764180 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 375/766] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 26 12:38:51 2022] PUT /identity/v3/projects/644278e7e8934b37a1e7ab20d35764cd/users/0f5d6f3c12244c0faf26048513dc49e7/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:38:51.764180 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a47eae-2270-46ea-8c6a-0b520849a436 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:51.904569 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ffcdd-81e4-4016-b415-4d5df06ba972 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:51.952077 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a47eae-2270-46ea-8c6a-0b520849a436 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:51.972012 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ed8723-ad7c-4476-89ec-33f7a369ff8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:51.973000 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed8723-ad7c-4476-89ec-33f7a369ff8f None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:51.973544 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed8723-ad7c-4476-89ec-33f7a369ff8f None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:51.974132 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed8723-ad7c-4476-89ec-33f7a369ff8f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:51.975567 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ed8723-ad7c-4476-89ec-33f7a369ff8f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:51.977377 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ed8723-ad7c-4476-89ec-33f7a369ff8f None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:51.987949 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-a0ed8723-ad7c-4476-89ec-33f7a369ff8f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 12:38:51.987949 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:51.987949 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 12:38:51.987949 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 12:38:51.987949 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 12:38:51.987949 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 12:38:51.987949 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 12:38:51.987949 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:38:51.987949 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 12:38:51.987949 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:51.987949 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 12:38:51.987949 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:51.987949 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:51.987949 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:51.987949 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:51.987949 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:51.988920 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:51.988920 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:51.988920 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:51.988920 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:51.988920 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:51.988920 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:51.988920 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:51.988920 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 12:38:51.988920 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 12:38:51.988920 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 12:38:51.988920 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 12:38:51.988920 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:51.988920 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:51.988920 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:51.988920 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:51.988920 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:51.990133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:51.990133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:51.990133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:51.990133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:51.990133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:51.990133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:51.990133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:51.990133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:51.990133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:51.990133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:51.990133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:51.990133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 12:38:51.990133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:38:51.990133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 12:38:51.990133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:51.993227 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 392/767] 10.222.0.20 () {56 vars in 1129 bytes} [Tue Jul 26 12:38:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:51.993347 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e512d592-5cd0-47ca-a6d5-b8da021be3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:51.994053 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512d592-5cd0-47ca-a6d5-b8da021be3c8 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:51.994227 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512d592-5cd0-47ca-a6d5-b8da021be3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:51.994485 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512d592-5cd0-47ca-a6d5-b8da021be3c8 None admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:51.995244 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e512d592-5cd0-47ca-a6d5-b8da021be3c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:51.995552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e512d592-5cd0-47ca-a6d5-b8da021be3c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:51.997041 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e512d592-5cd0-47ca-a6d5-b8da021be3c8 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:51.998965 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ed8723-ad7c-4476-89ec-33f7a369ff8f None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:52.012350 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 376/768] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 26 12:38:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:52.017992 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e512d592-5cd0-47ca-a6d5-b8da021be3c8 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:52.036111 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:52.080612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:52.131663 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ed8723-ad7c-4476-89ec-33f7a369ff8f None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:52.175664 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822fd81-713d-44e2-9907-73fbd106527b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:52.175664 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822fd81-713d-44e2-9907-73fbd106527b None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:52.175664 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822fd81-713d-44e2-9907-73fbd106527b None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:52.175664 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822fd81-713d-44e2-9907-73fbd106527b None admin] PATH_INFO: `/v3/services` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:52.176081 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9822fd81-713d-44e2-9907-73fbd106527b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:52.176081 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9822fd81-713d-44e2-9907-73fbd106527b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:52.176081 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9822fd81-713d-44e2-9907-73fbd106527b None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:52.184640 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 393/769] 10.222.0.20 () {56 vars in 1124 bytes} [Tue Jul 26 12:38:51 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:38:52.186993 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822fd81-713d-44e2-9907-73fbd106527b None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:52.207718 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e512d592-5cd0-47ca-a6d5-b8da021be3c8 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:52.248712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b151bb27-3026-479a-a494-e7cb0c563ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:52.249340 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b151bb27-3026-479a-a494-e7cb0c563ba5 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:52.249617 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b151bb27-3026-479a-a494-e7cb0c563ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:52.249887 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b151bb27-3026-479a-a494-e7cb0c563ba5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:52.250682 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b151bb27-3026-479a-a494-e7cb0c563ba5 None admin] RBAC: Adding query filter params (role.id=9c68629183974be9bfb1ed95f69912ee, scope.project.id=644278e7e8934b37a1e7ab20d35764cd, user.id=30fc532f464843a3ad35e4d3c387bb3a) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:52.250989 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b151bb27-3026-479a-a494-e7cb0c563ba5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:52.252462 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b151bb27-3026-479a-a494-e7cb0c563ba5 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:52.261783 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 377/770] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:38:52 2022] GET /identity/v3/role_assignments?user.id=30fc532f464843a3ad35e4d3c387bb3a&scope.project.id=644278e7e8934b37a1e7ab20d35764cd&role.id=9c68629183974be9bfb1ed95f69912ee => generated 259 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:38:52.263928 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d3c683-2314-47f7-b3e1-7316b7c2392d None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:52.264223 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d3c683-2314-47f7-b3e1-7316b7c2392d None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:52.264501 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d3c683-2314-47f7-b3e1-7316b7c2392d None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:52.266174 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 378/771] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:52.273077 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b8d6d1-b8af-44c9-a8c4-b5764732fcea None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:52.273430 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b8d6d1-b8af-44c9-a8c4-b5764732fcea None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:52.273823 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b8d6d1-b8af-44c9-a8c4-b5764732fcea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:52.294917 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-78b8d6d1-b8af-44c9-a8c4-b5764732fcea None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:52.295517 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-78b8d6d1-b8af-44c9-a8c4-b5764732fcea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:52.329096 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822fd81-713d-44e2-9907-73fbd106527b None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:52.355482 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:52.365682 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:52.371165 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b1e614-922f-4a25-a48e-4a8a292bb5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:52.372093 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1e614-922f-4a25-a48e-4a8a292bb5d0 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:52.372522 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1e614-922f-4a25-a48e-4a8a292bb5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:52.372942 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1e614-922f-4a25-a48e-4a8a292bb5d0 None admin] PATH_INFO: `/v3/services` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:52.373775 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b1e614-922f-4a25-a48e-4a8a292bb5d0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:52.374290 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b1e614-922f-4a25-a48e-4a8a292bb5d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:52.375403 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 379/772] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:38:52.376104 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b1e614-922f-4a25-a48e-4a8a292bb5d0 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:52.382435 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 394/773] 10.222.0.20 () {56 vars in 1124 bytes} [Tue Jul 26 12:38:52 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:52.389040 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b8d6d1-b8af-44c9-a8c4-b5764732fcea None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:52.413670 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b1e614-922f-4a25-a48e-4a8a292bb5d0 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:52.541043 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b1e614-922f-4a25-a48e-4a8a292bb5d0 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:52.567099 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b8d6d1-b8af-44c9-a8c4-b5764732fcea None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:52.600510 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b8e54a-a860-4aae-a848-aa0632affc8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:52.602443 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8e54a-a860-4aae-a848-aa0632affc8d None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:52.606133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8e54a-a860-4aae-a848-aa0632affc8d None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:52.606637 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8e54a-a860-4aae-a848-aa0632affc8d None admin] PATH_INFO: `/v3/services/network` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:52.607504 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b8e54a-a860-4aae-a848-aa0632affc8d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:52.609134 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b8e54a-a860-4aae-a848-aa0632affc8d None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:52.623504 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-63b8e54a-a860-4aae-a848-aa0632affc8d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 12:38:52.623504 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:52.623504 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 12:38:52.623504 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 12:38:52.623504 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 12:38:52.623504 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 12:38:52.623504 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 12:38:52.623504 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:38:52.623504 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 12:38:52.623504 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:52.623504 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 12:38:52.623504 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:52.623504 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:52.623504 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:52.623504 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:52.623504 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:52.623504 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:52.624780 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:52.624780 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:52.624780 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:52.624780 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:52.624780 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:52.624780 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:52.624780 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 12:38:52.624780 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 12:38:52.624780 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 12:38:52.624780 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 12:38:52.624780 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:52.624780 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:52.624780 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:52.624780 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:52.624780 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:52.624780 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:52.625795 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:52.625795 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:52.625795 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:52.625795 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:52.625795 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:52.625795 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:52.625795 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:52.625795 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:52.625795 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:52.625795 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:52.625795 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 12:38:52.625795 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:38:52.625795 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 12:38:52.625795 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:52.631660 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 395/774] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:38:52 2022] GET /identity/v3/services/network => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:38:52.638046 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b8e54a-a860-4aae-a848-aa0632affc8d None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:52.640145 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce95e197-84a9-4ba9-82cc-5fd063ee19a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:52.641326 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95e197-84a9-4ba9-82cc-5fd063ee19a8 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:52.643665 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95e197-84a9-4ba9-82cc-5fd063ee19a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:52.644097 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95e197-84a9-4ba9-82cc-5fd063ee19a8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:52.645020 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce95e197-84a9-4ba9-82cc-5fd063ee19a8 None admin] RBAC: Adding query filter params (interface=public, service_id=f8f9e522b4a14a719c06856aeec264ef, region_id=RegionOne) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:52.646339 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce95e197-84a9-4ba9-82cc-5fd063ee19a8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:52.647915 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce95e197-84a9-4ba9-82cc-5fd063ee19a8 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:52.665117 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 380/775] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Jul 26 12:38:52 2022] GET /identity/v3/endpoints?service_id=f8f9e522b4a14a719c06856aeec264ef&interface=public®ion_id=RegionOne => generated 194 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:52.676528 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce95e197-84a9-4ba9-82cc-5fd063ee19a8 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:52.704647 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:52.795489 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b8e54a-a860-4aae-a848-aa0632affc8d None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:52.796840 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:52.858736 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0e6c3-dd2f-4a6a-8d44-c53e3e9f90fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:52.859356 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0e6c3-dd2f-4a6a-8d44-c53e3e9f90fa None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:52.859654 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0e6c3-dd2f-4a6a-8d44-c53e3e9f90fa None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:52.859931 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0e6c3-dd2f-4a6a-8d44-c53e3e9f90fa None admin] PATH_INFO: `/v3/services` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:52.860615 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d0e6c3-dd2f-4a6a-8d44-c53e3e9f90fa None admin] RBAC: Adding query filter params (name=network) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:52.860922 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d0e6c3-dd2f-4a6a-8d44-c53e3e9f90fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:52.862292 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d0e6c3-dd2f-4a6a-8d44-c53e3e9f90fa None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:52.866721 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce95e197-84a9-4ba9-82cc-5fd063ee19a8 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:52.873278 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 396/776] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Jul 26 12:38:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:52.879074 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0e6c3-dd2f-4a6a-8d44-c53e3e9f90fa None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:52.930478 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba2169-9b90-4ab8-a7d2-76b55f5149df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:52.931108 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba2169-9b90-4ab8-a7d2-76b55f5149df None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:52.931392 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba2169-9b90-4ab8-a7d2-76b55f5149df None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:52.931687 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba2169-9b90-4ab8-a7d2-76b55f5149df None admin] PATH_INFO: `/v3/services` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:52.932353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfba2169-9b90-4ab8-a7d2-76b55f5149df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:52.933732 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfba2169-9b90-4ab8-a7d2-76b55f5149df None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:52.948329 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 381/777] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Jul 26 12:38:52 2022] GET /identity/v3/services => generated 772 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:53.040723 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0e6c3-dd2f-4a6a-8d44-c53e3e9f90fa None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:53.062252 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:53.085659 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c146a-aa47-4807-bd19-2e7018c77d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:53.086307 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c146a-aa47-4807-bd19-2e7018c77d73 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:53.086615 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c146a-aa47-4807-bd19-2e7018c77d73 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:53.086894 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c146a-aa47-4807-bd19-2e7018c77d73 None admin] PATH_INFO: `/v3/services` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:53.087606 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1c146a-aa47-4807-bd19-2e7018c77d73 None admin] RBAC: Adding query filter params (type=network) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:53.087982 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1c146a-aa47-4807-bd19-2e7018c77d73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:53.089392 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1c146a-aa47-4807-bd19-2e7018c77d73 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:53.099030 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 397/778] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Jul 26 12:38:52 2022] GET /identity/v3/services?type=network => generated 350 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:38:53.105385 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba2169-9b90-4ab8-a7d2-76b55f5149df None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:53.261042 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba2169-9b90-4ab8-a7d2-76b55f5149df None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:53.289730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:53.298767 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be56a9dd-c25f-47c5-bb37-d25e1ad69576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:53.299757 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be56a9dd-c25f-47c5-bb37-d25e1ad69576 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:53.300247 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be56a9dd-c25f-47c5-bb37-d25e1ad69576 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:53.300726 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be56a9dd-c25f-47c5-bb37-d25e1ad69576 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:53.301687 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be56a9dd-c25f-47c5-bb37-d25e1ad69576 None admin] RBAC: Adding query filter params (interface=public, service_id=cfa8768831674ff8bec1dd77ea8a1c17, region_id=RegionOne) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:53.302203 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be56a9dd-c25f-47c5-bb37-d25e1ad69576 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:53.303954 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be56a9dd-c25f-47c5-bb37-d25e1ad69576 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:53.310119 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 382/779] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Jul 26 12:38:53 2022] GET /identity/v3/endpoints?service_id=cfa8768831674ff8bec1dd77ea8a1c17&interface=public®ion_id=RegionOne => generated 194 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:53.315588 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c146a-aa47-4807-bd19-2e7018c77d73 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:53.432044 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c146a-aa47-4807-bd19-2e7018c77d73 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:53.481334 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3d9a4b-b28d-486b-9e3a-1e02483f2cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:53.482334 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d9a4b-b28d-486b-9e3a-1e02483f2cc8 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:53.482822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d9a4b-b28d-486b-9e3a-1e02483f2cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:53.483299 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d9a4b-b28d-486b-9e3a-1e02483f2cc8 None admin] PATH_INFO: `/v3/services` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:53.484191 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3d9a4b-b28d-486b-9e3a-1e02483f2cc8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:53.485766 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3d9a4b-b28d-486b-9e3a-1e02483f2cc8 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:53.496131 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 398/780] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Jul 26 12:38:53 2022] GET /identity/v3/services => generated 772 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:53.660851 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254ad6d-27f2-491f-bcc9-484a9c08ce7a None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:53.661960 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254ad6d-27f2-491f-bcc9-484a9c08ce7a None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:53.662463 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254ad6d-27f2-491f-bcc9-484a9c08ce7a None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:53.664504 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 383/781] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:53.669869 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77ba5ca-90d7-4d78-bbbe-2ef1fb0f82fa None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:53.670787 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77ba5ca-90d7-4d78-bbbe-2ef1fb0f82fa None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:53.671294 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77ba5ca-90d7-4d78-bbbe-2ef1fb0f82fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:53.697030 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-c77ba5ca-90d7-4d78-bbbe-2ef1fb0f82fa None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:53.698616 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-c77ba5ca-90d7-4d78-bbbe-2ef1fb0f82fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:53.741982 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:53.777345 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 399/782] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:38:53.800300 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254ad6d-27f2-491f-bcc9-484a9c08ce7a None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:53.932622 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254ad6d-27f2-491f-bcc9-484a9c08ce7a None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:53.977061 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba8072-60ca-437f-9172-d64db75652be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:53.978106 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba8072-60ca-437f-9172-d64db75652be None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:53.978595 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba8072-60ca-437f-9172-d64db75652be None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:53.979080 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba8072-60ca-437f-9172-d64db75652be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:53.990323 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba8072-60ca-437f-9172-d64db75652be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:38:53.995784 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-52ba8072-60ca-437f-9172-d64db75652be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:38:53.995784 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:53.995784 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:53.995784 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:53.995784 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:53.995784 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:53.995784 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:53.995784 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:53.995784 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:53.995784 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:53.995784 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:53.995784 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:53.995784 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:38:53.995784 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:38:53.995784 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:38:53.995784 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:38:53.996708 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:38:53.996708 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:38:53.996708 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:38:53.996708 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:38:53.996708 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:53.996708 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:53.996708 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:53.996708 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:53.996708 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:53.996708 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:53.996708 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:53.996708 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:53.996708 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:53.996708 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:53.996708 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:53.996708 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:53.997540 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:53.997540 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:53.997540 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:53.997540 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:53.997540 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:38:53.997540 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:38:53.997540 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:38:53.997540 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:38:53.997540 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:38:53.997540 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:38:53.997540 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:38:53.997540 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:54.000250 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 384/783] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 26 12:38:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:54.008753 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77ba5ca-90d7-4d78-bbbe-2ef1fb0f82fa None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:54.146118 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77ba5ca-90d7-4d78-bbbe-2ef1fb0f82fa None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:54.184597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a13ab-3d82-4315-b422-e1997c2bd61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:54.185162 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a13ab-3d82-4315-b422-e1997c2bd61e None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:54.185443 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a13ab-3d82-4315-b422-e1997c2bd61e None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:54.185717 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a13ab-3d82-4315-b422-e1997c2bd61e None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:54.186436 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a13ab-3d82-4315-b422-e1997c2bd61e None admin] RBAC: Adding query filter params (name=admin) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:54.186741 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a13ab-3d82-4315-b422-e1997c2bd61e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:54.188152 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a13ab-3d82-4315-b422-e1997c2bd61e None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:54.197756 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 400/784] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:38:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:54.203162 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba8072-60ca-437f-9172-d64db75652be None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:54.299543 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:54.303681 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7edd27-dea0-4a13-83d1-5cdd6f3b425a None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:54.303974 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7edd27-dea0-4a13-83d1-5cdd6f3b425a None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:54.304249 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7edd27-dea0-4a13-83d1-5cdd6f3b425a None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:54.306540 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 401/785] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:54.313162 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58efbd-8548-473e-8550-4b9213287b18 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:54.313448 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58efbd-8548-473e-8550-4b9213287b18 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:54.313727 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58efbd-8548-473e-8550-4b9213287b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:54.353284 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-6a58efbd-8548-473e-8550-4b9213287b18 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:54.354573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-6a58efbd-8548-473e-8550-4b9213287b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:54.386417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba8072-60ca-437f-9172-d64db75652be None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:54.424143 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c375dd02-13c2-4996-a491-3ad116790c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:54.424798 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375dd02-13c2-4996-a491-3ad116790c7d None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:54.425080 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375dd02-13c2-4996-a491-3ad116790c7d None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:54.425357 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375dd02-13c2-4996-a491-3ad116790c7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:54.433033 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:54.439581 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c375dd02-13c2-4996-a491-3ad116790c7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:54.441016 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c375dd02-13c2-4996-a491-3ad116790c7d None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:54.466764 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-c375dd02-13c2-4996-a491-3ad116790c7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:54.466764 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:54.466764 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:54.466764 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:54.466764 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:54.466764 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:54.466764 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:54.466764 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:54.466764 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:54.466764 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:54.466764 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:54.466764 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:54.466764 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:54.466764 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:54.466764 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:54.467721 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:54.467721 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:54.467721 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:54.467721 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:54.467721 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:54.467721 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:54.467721 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:54.467721 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:54.467721 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:54.467721 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:54.467721 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:54.467721 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:54.467721 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:54.467721 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:54.467721 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:54.467721 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:54.467721 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:54.468629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:54.468629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:54.468629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:54.468629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:54.473244 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 385/786] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:54.478381 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c375dd02-13c2-4996-a491-3ad116790c7d None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:54.552345 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 402/787] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 12:38:54.588734 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a58efbd-8548-473e-8550-4b9213287b18 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:54.604518 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c375dd02-13c2-4996-a491-3ad116790c7d None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:54.608020 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:54.649797 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb83583-db5e-4c95-84ef-16d7702d0914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:54.650484 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb83583-db5e-4c95-84ef-16d7702d0914 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:54.650769 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb83583-db5e-4c95-84ef-16d7702d0914 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:54.651098 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb83583-db5e-4c95-84ef-16d7702d0914 None admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:54.651823 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb83583-db5e-4c95-84ef-16d7702d0914 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:54.652133 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb83583-db5e-4c95-84ef-16d7702d0914 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:54.653521 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb83583-db5e-4c95-84ef-16d7702d0914 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:54.687050 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 386/788] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:54.688029 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb83583-db5e-4c95-84ef-16d7702d0914 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:54.770779 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a58efbd-8548-473e-8550-4b9213287b18 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:54.810047 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb57169-3d08-4bf3-b858-240022038564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:54.810835 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb57169-3d08-4bf3-b858-240022038564 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:54.811121 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb57169-3d08-4bf3-b858-240022038564 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:54.811395 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb57169-3d08-4bf3-b858-240022038564 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:54.823162 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb57169-3d08-4bf3-b858-240022038564 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:38:54.824700 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb83583-db5e-4c95-84ef-16d7702d0914 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:54.828407 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-abb57169-3d08-4bf3-b858-240022038564 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:38:54.828407 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:54.828407 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:54.828407 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:54.828407 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:54.828407 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:54.828407 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:54.828407 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:54.828407 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:54.828407 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:54.828407 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:54.828407 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:54.828407 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:38:54.828407 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:38:54.828407 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:38:54.828407 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:38:54.829307 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:38:54.829307 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:38:54.829307 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:38:54.829307 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:38:54.829307 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:54.829307 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:54.829307 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:54.829307 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:54.829307 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:54.829307 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:54.829307 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:54.829307 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:54.829307 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:54.829307 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:54.829307 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:54.829307 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:54.830769 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:54.830769 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:54.830769 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:54.830769 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:54.830769 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:38:54.830769 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:38:54.830769 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:38:54.830769 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:38:54.830769 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:38:54.830769 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:38:54.830769 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:38:54.830769 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:54.830769 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 403/789] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 26 12:38:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:54.831482 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e9e57-c879-4d9f-88e7-5ec8a58a44e6 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:54.831482 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e9e57-c879-4d9f-88e7-5ec8a58a44e6 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:54.831828 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e9e57-c879-4d9f-88e7-5ec8a58a44e6 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:54.837263 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 404/790] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:54.838142 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52e9e57-c879-4d9f-88e7-5ec8a58a44e6 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:54.864012 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bd30dc-fcd5-4455-87b2-96b9e9f4a928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:54.865246 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd30dc-fcd5-4455-87b2-96b9e9f4a928 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:54.865530 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd30dc-fcd5-4455-87b2-96b9e9f4a928 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:54.865804 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd30dc-fcd5-4455-87b2-96b9e9f4a928 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:54.867189 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:54.875011 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bd30dc-fcd5-4455-87b2-96b9e9f4a928 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:54.877228 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bd30dc-fcd5-4455-87b2-96b9e9f4a928 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:54.892911 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-12bd30dc-fcd5-4455-87b2-96b9e9f4a928 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 12:38:54.892911 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:54.892911 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:54.892911 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:54.892911 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:54.892911 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:54.892911 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:54.892911 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:54.892911 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:54.892911 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:54.892911 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:54.892911 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:54.892911 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 12:38:54.892911 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 12:38:54.892911 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 12:38:54.892911 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 12:38:54.894887 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:54.894887 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:54.894887 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 12:38:54.894887 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:54.894887 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 12:38:54.894887 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:54.894887 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:54.894887 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:54.894887 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:54.894887 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:54.894887 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:54.894887 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:54.894887 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:54.894887 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:54.894887 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:54.894887 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:54.894887 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:54.894887 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:54.895960 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:54.895960 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:54.895960 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 12:38:54.895960 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 12:38:54.895960 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 12:38:54.895960 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 12:38:54.895960 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 12:38:54.895960 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 12:38:54.895960 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 12:38:54.895960 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:54.895960 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 387/791] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 26 12:38:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:54.895960 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450c873-a64e-4fc7-8968-6cd980a5022a None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:54.896751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450c873-a64e-4fc7-8968-6cd980a5022a None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:54.896751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450c873-a64e-4fc7-8968-6cd980a5022a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:54.918660 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-0450c873-a64e-4fc7-8968-6cd980a5022a None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:54.920587 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-0450c873-a64e-4fc7-8968-6cd980a5022a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:54.944024 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:54.968484 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52e9e57-c879-4d9f-88e7-5ec8a58a44e6 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:55.010207 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 388/792] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:38:55.011251 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0450c873-a64e-4fc7-8968-6cd980a5022a None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:55.022591 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9752d589-98fd-44ff-94c3-80fe9398d667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:55.023157 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752d589-98fd-44ff-94c3-80fe9398d667 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:55.023443 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752d589-98fd-44ff-94c3-80fe9398d667 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:55.023746 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752d589-98fd-44ff-94c3-80fe9398d667 None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:55.026290 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9752d589-98fd-44ff-94c3-80fe9398d667 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:55.026603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9752d589-98fd-44ff-94c3-80fe9398d667 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:55.030076 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9752d589-98fd-44ff-94c3-80fe9398d667 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:55.043834 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 405/793] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:38:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:55.044868 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9752d589-98fd-44ff-94c3-80fe9398d667 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:55.138848 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0450c873-a64e-4fc7-8968-6cd980a5022a None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:55.158685 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:55.178342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9752d589-98fd-44ff-94c3-80fe9398d667 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:55.189912 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c3cb0-e413-41c4-b559-ef3fee3f64cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:55.190249 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c3cb0-e413-41c4-b559-ef3fee3f64cb None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:55.190592 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c3cb0-e413-41c4-b559-ef3fee3f64cb None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:55.191021 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c3cb0-e413-41c4-b559-ef3fee3f64cb None admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:55.191932 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651c3cb0-e413-41c4-b559-ef3fee3f64cb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:55.192307 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651c3cb0-e413-41c4-b559-ef3fee3f64cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:55.194026 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651c3cb0-e413-41c4-b559-ef3fee3f64cb None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:55.219119 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 389/794] 10.222.0.20 () {56 vars in 1135 bytes} [Tue Jul 26 12:38:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:55.220449 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c3cb0-e413-41c4-b559-ef3fee3f64cb None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:55.222395 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a4e078-7d3f-4d79-817e-048bd6bc4fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:55.223115 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a4e078-7d3f-4d79-817e-048bd6bc4fca None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:55.223781 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a4e078-7d3f-4d79-817e-048bd6bc4fca None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:55.224136 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a4e078-7d3f-4d79-817e-048bd6bc4fca None admin] PATH_INFO: `/v3/roles/service` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:55.237638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:55.239096 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a4e078-7d3f-4d79-817e-048bd6bc4fca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:38:55.247730 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-58a4e078-7d3f-4d79-817e-048bd6bc4fca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 12:38:55.247730 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:55.247730 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:55.247730 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:55.247730 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:55.247730 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:55.247730 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:55.247730 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:55.247730 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:55.247730 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:55.247730 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:55.247730 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:55.247730 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:38:55.247730 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:38:55.247730 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:38:55.247730 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:38:55.249236 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:38:55.249236 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:38:55.249236 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:38:55.249236 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:38:55.249236 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:55.249236 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:55.249236 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:55.249236 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:55.249236 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:55.249236 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:55.249236 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:55.249236 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:55.249236 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:55.249236 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:55.249236 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:55.249236 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:55.250074 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:55.250074 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:55.250074 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:55.250074 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:55.250074 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:38:55.250074 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:38:55.250074 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:38:55.250074 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:38:55.250074 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:38:55.250074 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:38:55.250074 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 12:38:55.250074 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:55.250074 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 406/795] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 26 12:38:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:55.251006 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a4e078-7d3f-4d79-817e-048bd6bc4fca None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:55.359239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c3cb0-e413-41c4-b559-ef3fee3f64cb None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:55.379952 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a4e078-7d3f-4d79-817e-048bd6bc4fca None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:55.406880 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58acc0f8-faee-44f6-9837-7c4f84d770ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:55.407530 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58acc0f8-faee-44f6-9837-7c4f84d770ab None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:55.407843 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58acc0f8-faee-44f6-9837-7c4f84d770ab None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:55.408391 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58acc0f8-faee-44f6-9837-7c4f84d770ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:55.417136 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f9e389-17a0-4361-9761-b08346509f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:55.417982 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9e389-17a0-4361-9761-b08346509f45 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:55.418374 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9e389-17a0-4361-9761-b08346509f45 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:55.418787 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9e389-17a0-4361-9761-b08346509f45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:55.420956 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58acc0f8-faee-44f6-9837-7c4f84d770ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:55.423059 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58acc0f8-faee-44f6-9837-7c4f84d770ab None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:55.436704 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f9e389-17a0-4361-9761-b08346509f45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:55.438225 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f9e389-17a0-4361-9761-b08346509f45 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:55.440749 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-58acc0f8-faee-44f6-9837-7c4f84d770ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:55.440749 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:55.440749 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:55.440749 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:55.440749 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:55.440749 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:55.440749 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:55.440749 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:55.440749 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:55.440749 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:55.440749 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:55.440749 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:55.440749 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:55.440749 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:55.440749 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:55.442368 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:55.442368 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:55.442368 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:55.442368 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:55.442368 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:55.442368 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:55.442368 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:55.442368 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:55.442368 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:55.442368 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:55.442368 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:55.442368 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:55.442368 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:55.442368 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:55.442368 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:55.442368 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:55.442368 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:55.444538 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:55.444538 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:55.444538 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:55.444538 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:55.444538 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 390/796] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:55.444538 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58acc0f8-faee-44f6-9837-7c4f84d770ab None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:55.445283 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:55.456769 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-98f9e389-17a0-4361-9761-b08346509f45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:55.456769 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:55.456769 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:55.456769 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:55.456769 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:55.456769 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:55.456769 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:55.456769 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:55.456769 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:55.456769 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:55.456769 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:55.456769 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:55.456769 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:55.456769 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:55.456769 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:55.457922 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:55.457922 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:55.457922 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:55.457922 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:55.457922 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:55.457922 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:55.457922 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:55.457922 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:55.457922 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:55.457922 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:55.457922 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:55.457922 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:55.457922 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:55.457922 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:55.457922 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:55.457922 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:55.457922 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:55.459053 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:55.459053 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:55.459053 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:55.459053 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:55.461232 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 407/797] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:55.462031 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f9e389-17a0-4361-9761-b08346509f45 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:55.534322 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:55.576969 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f9e389-17a0-4361-9761-b08346509f45 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:55.581778 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58acc0f8-faee-44f6-9837-7c4f84d770ab None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:55.616134 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4e518-f496-4dc8-b9e7-11b441c145c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:55.616688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4e518-f496-4dc8-b9e7-11b441c145c9 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:55.616970 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4e518-f496-4dc8-b9e7-11b441c145c9 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:55.617242 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4e518-f496-4dc8-b9e7-11b441c145c9 None admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:55.617945 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b4e518-f496-4dc8-b9e7-11b441c145c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:55.618246 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b4e518-f496-4dc8-b9e7-11b441c145c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:55.619629 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b4e518-f496-4dc8-b9e7-11b441c145c9 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:55.627410 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affbabae-5cf9-4969-94a9-7ddf931cb4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:55.628082 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affbabae-5cf9-4969-94a9-7ddf931cb4bf None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:55.628365 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affbabae-5cf9-4969-94a9-7ddf931cb4bf None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:55.628679 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affbabae-5cf9-4969-94a9-7ddf931cb4bf None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:55.630010 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affbabae-5cf9-4969-94a9-7ddf931cb4bf None admin] RBAC: Adding query filter params (name=service) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:55.630670 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affbabae-5cf9-4969-94a9-7ddf931cb4bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:55.638052 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affbabae-5cf9-4969-94a9-7ddf931cb4bf None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:55.638222 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 408/798] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:55.638222 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4e518-f496-4dc8-b9e7-11b441c145c9 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:55.645645 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 391/799] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Jul 26 12:38:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:55.647714 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8704cf-6251-43bf-b163-09fd48080659 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:55.647714 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8704cf-6251-43bf-b163-09fd48080659 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:55.647714 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8704cf-6251-43bf-b163-09fd48080659 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:55.650646 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 392/800] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:55.650646 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8704cf-6251-43bf-b163-09fd48080659 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:55.698697 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:55.742400 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4e518-f496-4dc8-b9e7-11b441c145c9 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:55.764934 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8704cf-6251-43bf-b163-09fd48080659 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:55.778641 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d337769-8358-4f79-8319-fcdacd950adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:55.779328 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d337769-8358-4f79-8319-fcdacd950adf None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:55.779534 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d337769-8358-4f79-8319-fcdacd950adf None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:55.779832 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d337769-8358-4f79-8319-fcdacd950adf None admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:55.780520 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d337769-8358-4f79-8319-fcdacd950adf None admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:55.780824 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d337769-8358-4f79-8319-fcdacd950adf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:55.782189 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d337769-8358-4f79-8319-fcdacd950adf None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:55.792415 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:55.794498 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 409/801] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:55.795300 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d337769-8358-4f79-8319-fcdacd950adf None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:55.808704 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de2aba8-ee12-490c-94fa-9c50e1cb964b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:55.809513 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de2aba8-ee12-490c-94fa-9c50e1cb964b None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:55.809859 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de2aba8-ee12-490c-94fa-9c50e1cb964b None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:55.810130 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de2aba8-ee12-490c-94fa-9c50e1cb964b None admin] PATH_INFO: `/v3/users/nova` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:55.819857 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de2aba8-ee12-490c-94fa-9c50e1cb964b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:55.821309 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de2aba8-ee12-490c-94fa-9c50e1cb964b None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:55.835930 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-1de2aba8-ee12-490c-94fa-9c50e1cb964b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 12:38:55.835930 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:55.835930 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:55.835930 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:55.835930 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:55.835930 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:55.835930 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:55.835930 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:55.835930 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:55.835930 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:55.835930 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:55.835930 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:55.835930 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 12:38:55.835930 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 12:38:55.835930 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 12:38:55.835930 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 12:38:55.836937 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:55.836937 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:55.836937 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 12:38:55.836937 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:55.836937 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 12:38:55.836937 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:55.836937 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:55.836937 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:55.836937 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:55.836937 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:55.836937 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:55.836937 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:55.836937 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:55.836937 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:55.836937 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:55.836937 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:55.836937 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:55.836937 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:55.837875 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:55.837875 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:55.837875 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 12:38:55.837875 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 12:38:55.837875 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 12:38:55.837875 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 12:38:55.837875 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 12:38:55.837875 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 12:38:55.837875 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 12:38:55.837875 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:55.837875 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 393/802] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Jul 26 12:38:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:55.838493 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ceba2c-f6f3-4c0d-92c4-0490903d800c None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:55.839942 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ceba2c-f6f3-4c0d-92c4-0490903d800c None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:55.839942 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ceba2c-f6f3-4c0d-92c4-0490903d800c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:55.859105 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-58ceba2c-f6f3-4c0d-92c4-0490903d800c None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:55.860332 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-58ceba2c-f6f3-4c0d-92c4-0490903d800c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:55.916921 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d337769-8358-4f79-8319-fcdacd950adf None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:55.943835 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 394/803] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:38:55.943835 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ceba2c-f6f3-4c0d-92c4-0490903d800c None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:55.945226 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:55.966570 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da261cc9-e06c-4459-a8b1-b56e269852b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:55.967271 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da261cc9-e06c-4459-a8b1-b56e269852b5 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:55.967560 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da261cc9-e06c-4459-a8b1-b56e269852b5 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:55.968008 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da261cc9-e06c-4459-a8b1-b56e269852b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:55.979114 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da261cc9-e06c-4459-a8b1-b56e269852b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:55.980623 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da261cc9-e06c-4459-a8b1-b56e269852b5 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:55.996310 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-da261cc9-e06c-4459-a8b1-b56e269852b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:55.996310 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:55.996310 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:55.996310 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:55.996310 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:55.996310 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:55.996310 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:55.996310 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:55.996310 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:55.996310 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:55.996310 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:55.996310 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:55.996310 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:55.996310 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:55.996310 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:55.997919 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:55.997919 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:55.997919 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:55.997919 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:55.997919 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:55.997919 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:55.997919 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:55.997919 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:55.997919 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:55.997919 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:55.997919 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:55.997919 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:55.997919 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:55.997919 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:55.997919 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:55.997919 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:55.997919 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:56.000209 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:56.000209 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:56.000209 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:56.000209 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:56.000209 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 410/804] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:56.000209 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da261cc9-e06c-4459-a8b1-b56e269852b5 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:56.062128 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ceba2c-f6f3-4c0d-92c4-0490903d800c None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:56.080438 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:56.099561 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d3714-f0b8-40e9-a3d2-4c9936ce2f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:56.100220 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d3714-f0b8-40e9-a3d2-4c9936ce2f6a None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:56.100499 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d3714-f0b8-40e9-a3d2-4c9936ce2f6a None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:56.100767 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d3714-f0b8-40e9-a3d2-4c9936ce2f6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:56.108433 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7d3714-f0b8-40e9-a3d2-4c9936ce2f6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:56.109874 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7d3714-f0b8-40e9-a3d2-4c9936ce2f6a None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:56.110478 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da261cc9-e06c-4459-a8b1-b56e269852b5 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:56.123487 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-6c7d3714-f0b8-40e9-a3d2-4c9936ce2f6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:56.123487 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:56.123487 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:56.123487 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:56.123487 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:56.123487 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:56.123487 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:56.123487 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:56.123487 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:56.123487 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:56.123487 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:56.123487 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:56.123487 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 12:38:56.123487 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 12:38:56.123487 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 12:38:56.124502 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 12:38:56.124502 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:56.124502 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:56.124502 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:56.124502 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:56.124502 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:56.124502 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:56.124502 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:56.124502 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:56.124502 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:56.124502 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:56.124502 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:56.124502 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:56.124502 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:56.124502 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:56.124502 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:56.124502 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:56.125376 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 12:38:56.125376 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 12:38:56.125376 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 12:38:56.125376 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 12:38:56.125376 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 12:38:56.125376 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 12:38:56.125376 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:56.125376 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:56.125376 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 395/805] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:38:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:56.125849 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d3714-f0b8-40e9-a3d2-4c9936ce2f6a None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:56.150151 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90979b1d-c4bb-47bc-9626-b2413e9cf712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:56.150766 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90979b1d-c4bb-47bc-9626-b2413e9cf712 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:56.151040 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90979b1d-c4bb-47bc-9626-b2413e9cf712 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:56.151310 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90979b1d-c4bb-47bc-9626-b2413e9cf712 None admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:56.152128 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90979b1d-c4bb-47bc-9626-b2413e9cf712 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:56.152434 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90979b1d-c4bb-47bc-9626-b2413e9cf712 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:56.153862 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90979b1d-c4bb-47bc-9626-b2413e9cf712 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:56.174648 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 411/806] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Jul 26 12:38:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:56.175414 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90979b1d-c4bb-47bc-9626-b2413e9cf712 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:56.204812 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:56.230054 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d3714-f0b8-40e9-a3d2-4c9936ce2f6a None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:56.270800 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e467273-64fc-4fb1-a3fc-a37295247a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:56.271935 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e467273-64fc-4fb1-a3fc-a37295247a49 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:56.272434 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e467273-64fc-4fb1-a3fc-a37295247a49 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:56.272921 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e467273-64fc-4fb1-a3fc-a37295247a49 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:56.274127 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e467273-64fc-4fb1-a3fc-a37295247a49 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:56.276639 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e467273-64fc-4fb1-a3fc-a37295247a49 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:56.279808 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90979b1d-c4bb-47bc-9626-b2413e9cf712 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:56.286086 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-3e467273-64fc-4fb1-a3fc-a37295247a49 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 12:38:56.286086 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:56.286086 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 12:38:56.286086 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 12:38:56.286086 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 12:38:56.286086 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 12:38:56.286086 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 12:38:56.286086 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:38:56.286086 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 12:38:56.286086 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:56.286086 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 12:38:56.286086 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:56.286086 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:56.286086 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:56.286086 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:56.286086 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:56.287559 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:56.287559 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:56.287559 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:56.287559 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:56.287559 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:56.287559 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:56.287559 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:56.287559 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 12:38:56.287559 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 12:38:56.287559 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 12:38:56.287559 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 12:38:56.287559 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:56.287559 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:56.287559 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:56.287559 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:56.287559 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:56.288431 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:56.288431 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:56.288431 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:56.288431 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:56.288431 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:56.288431 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:56.288431 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:56.288431 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:56.288431 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:56.288431 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:56.288431 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:56.288431 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 12:38:56.288431 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:38:56.288431 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 12:38:56.288431 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:56.288431 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 396/807] 10.222.0.20 () {56 vars in 1129 bytes} [Tue Jul 26 12:38:56 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:56.289273 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501469a1-cb41-4987-bbbe-e835a289a287 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:56.289798 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501469a1-cb41-4987-bbbe-e835a289a287 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:56.290031 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501469a1-cb41-4987-bbbe-e835a289a287 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:56.291252 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 397/808] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:56.291960 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501469a1-cb41-4987-bbbe-e835a289a287 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:56.320645 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a8fb7-9a05-4f2e-a42f-47ff81b0c1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:56.321263 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a8fb7-9a05-4f2e-a42f-47ff81b0c1f9 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:56.321542 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a8fb7-9a05-4f2e-a42f-47ff81b0c1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:56.321816 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a8fb7-9a05-4f2e-a42f-47ff81b0c1f9 None admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:56.322512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a8fb7-9a05-4f2e-a42f-47ff81b0c1f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:56.322819 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a8fb7-9a05-4f2e-a42f-47ff81b0c1f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:56.324255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a8fb7-9a05-4f2e-a42f-47ff81b0c1f9 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:56.334906 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:56.336943 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 412/809] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:56.337773 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a8fb7-9a05-4f2e-a42f-47ff81b0c1f9 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:56.398596 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501469a1-cb41-4987-bbbe-e835a289a287 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:56.430889 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:56.438498 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9161cd-7d84-4b10-880f-ad235cfd37aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:56.439111 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9161cd-7d84-4b10-880f-ad235cfd37aa None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:56.439387 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9161cd-7d84-4b10-880f-ad235cfd37aa None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:56.439676 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9161cd-7d84-4b10-880f-ad235cfd37aa None admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:56.440421 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9161cd-7d84-4b10-880f-ad235cfd37aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:56.440725 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9161cd-7d84-4b10-880f-ad235cfd37aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:56.442160 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9161cd-7d84-4b10-880f-ad235cfd37aa None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:56.445524 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a8fb7-9a05-4f2e-a42f-47ff81b0c1f9 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:56.453907 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 398/810] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 26 12:38:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:56.454722 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9161cd-7d84-4b10-880f-ad235cfd37aa None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:56.483406 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d040a55a-7e85-4a9b-a06e-76490996865c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:56.484043 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040a55a-7e85-4a9b-a06e-76490996865c None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:56.484319 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040a55a-7e85-4a9b-a06e-76490996865c None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:56.484590 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040a55a-7e85-4a9b-a06e-76490996865c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:56.556851 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:56.560265 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9161cd-7d84-4b10-880f-ad235cfd37aa None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:56.562698 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d040a55a-7e85-4a9b-a06e-76490996865c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:56.564248 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d040a55a-7e85-4a9b-a06e-76490996865c None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:56.599257 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b473cd-711b-4368-ba00-d3298cb91ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:56.600508 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b473cd-711b-4368-ba00-d3298cb91ead None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:56.600786 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b473cd-711b-4368-ba00-d3298cb91ead None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:56.601061 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b473cd-711b-4368-ba00-d3298cb91ead None admin] PATH_INFO: `/v3/services` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:56.601736 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b473cd-711b-4368-ba00-d3298cb91ead None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:56.602043 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b473cd-711b-4368-ba00-d3298cb91ead None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:56.603431 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b473cd-711b-4368-ba00-d3298cb91ead None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:56.608045 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 399/811] 10.222.0.20 () {56 vars in 1124 bytes} [Tue Jul 26 12:38:56 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:56.609227 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1182f7b-9f24-44fd-99e2-32ca28c3a730 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:56.609512 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1182f7b-9f24-44fd-99e2-32ca28c3a730 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:56.609785 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1182f7b-9f24-44fd-99e2-32ca28c3a730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:56.627326 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-e1182f7b-9f24-44fd-99e2-32ca28c3a730 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:56.628664 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-e1182f7b-9f24-44fd-99e2-32ca28c3a730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:56.657798 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:56.673479 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 413/812] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:38:56 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 12:38:56.674316 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d040a55a-7e85-4a9b-a06e-76490996865c None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:56.701870 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 400/813] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:38:56.701870 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1182f7b-9f24-44fd-99e2-32ca28c3a730 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:56.776858 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d040a55a-7e85-4a9b-a06e-76490996865c None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:56.809574 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:56.812738 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd8d97-6eaa-4302-811a-d07f86750663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:56.813416 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd8d97-6eaa-4302-811a-d07f86750663 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:56.813649 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd8d97-6eaa-4302-811a-d07f86750663 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:56.813928 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd8d97-6eaa-4302-811a-d07f86750663 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:56.817408 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1182f7b-9f24-44fd-99e2-32ca28c3a730 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:56.822804 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd8d97-6eaa-4302-811a-d07f86750663 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:56.824646 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd8d97-6eaa-4302-811a-d07f86750663 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:56.838818 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-e2bd8d97-6eaa-4302-811a-d07f86750663 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 12:38:56.838818 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:56.838818 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:56.838818 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:56.838818 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:56.838818 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:56.838818 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:56.838818 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:56.838818 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:56.838818 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:56.838818 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:56.838818 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:56.838818 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 12:38:56.838818 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 12:38:56.838818 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 12:38:56.838818 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 12:38:56.840050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:56.840050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:56.840050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 12:38:56.840050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:56.840050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 12:38:56.840050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:38:56.840050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:56.840050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:56.840050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:56.840050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:56.840050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:56.840050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:56.840050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:56.840050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:56.840050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:56.840050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:56.840050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:56.840050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:56.841002 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:56.841002 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:56.841002 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 12:38:56.841002 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 12:38:56.841002 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 12:38:56.841002 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 12:38:56.841002 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 12:38:56.841002 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 12:38:56.841002 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 12:38:56.841002 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:56.841002 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 414/814] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Jul 26 12:38:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:56.841002 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd8d97-6eaa-4302-811a-d07f86750663 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:56.857753 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb12ea4f-6549-441b-89b4-a85b831c4c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:56.858530 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12ea4f-6549-441b-89b4-a85b831c4c2e None admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:56.858810 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12ea4f-6549-441b-89b4-a85b831c4c2e None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:56.859081 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12ea4f-6549-441b-89b4-a85b831c4c2e None admin] PATH_INFO: `/v3/projects/644278e7e8934b37a1e7ab20d35764cd/users/cd2483f74d7d48b1a7b8ffe32f63abde/roles/0e78444e43c744668178971af4087ca0` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:56.861079 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb12ea4f-6549-441b-89b4-a85b831c4c2e None admin] RBAC: Authorizing `identity:create_grant(project_id=644278e7e8934b37a1e7ab20d35764cd, user_id=cd2483f74d7d48b1a7b8ffe32f63abde, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:56.862509 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb12ea4f-6549-441b-89b4-a85b831c4c2e None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:56.876094 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 401/815] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 26 12:38:56 2022] PUT /identity/v3/projects/644278e7e8934b37a1e7ab20d35764cd/users/cd2483f74d7d48b1a7b8ffe32f63abde/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:38:56.876694 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb12ea4f-6549-441b-89b4-a85b831c4c2e None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:56.894416 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:56.965722 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd8d97-6eaa-4302-811a-d07f86750663 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:56.999662 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb12ea4f-6549-441b-89b4-a85b831c4c2e None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:57.002030 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e81024d-3e74-40b5-8339-db9f6fc55674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:57.002662 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81024d-3e74-40b5-8339-db9f6fc55674 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:57.002941 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81024d-3e74-40b5-8339-db9f6fc55674 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:57.003215 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81024d-3e74-40b5-8339-db9f6fc55674 None admin] PATH_INFO: `/v3/services` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:57.003928 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e81024d-3e74-40b5-8339-db9f6fc55674 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:57.004233 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e81024d-3e74-40b5-8339-db9f6fc55674 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:57.006161 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e81024d-3e74-40b5-8339-db9f6fc55674 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:57.016450 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 415/816] 10.222.0.20 () {56 vars in 1124 bytes} [Tue Jul 26 12:38:56 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:57.016450 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e81024d-3e74-40b5-8339-db9f6fc55674 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:57.047121 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f9965-fc14-4e0e-9b90-736ab17dc8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:57.048064 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f9965-fc14-4e0e-9b90-736ab17dc8c2 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:57.048341 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f9965-fc14-4e0e-9b90-736ab17dc8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:57.048618 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f9965-fc14-4e0e-9b90-736ab17dc8c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:57.059489 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69f9965-fc14-4e0e-9b90-736ab17dc8c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:57.061078 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69f9965-fc14-4e0e-9b90-736ab17dc8c2 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:57.077312 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-f69f9965-fc14-4e0e-9b90-736ab17dc8c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:57.077312 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:57.077312 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:57.077312 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:57.077312 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:57.077312 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:57.077312 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:57.077312 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:57.077312 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:57.077312 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:57.077312 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:57.077312 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:57.077312 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:57.077312 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:57.077312 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:57.078423 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:57.078423 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:57.078423 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:57.078423 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:57.078423 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:57.078423 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:57.078423 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:57.078423 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:57.078423 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:57.078423 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:57.078423 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:57.078423 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:57.078423 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:57.078423 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:57.078423 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:57.078423 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:57.078423 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:57.079652 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:57.079652 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:57.079652 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:57.079652 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:57.079652 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 402/817] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:57.079652 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f9965-fc14-4e0e-9b90-736ab17dc8c2 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:57.094091 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:57.135678 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e81024d-3e74-40b5-8339-db9f6fc55674 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:57.150710 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:57.176284 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a8ebc-bfba-46f5-a145-225df1b8b99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:57.176934 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a8ebc-bfba-46f5-a145-225df1b8b99e None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:57.177214 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a8ebc-bfba-46f5-a145-225df1b8b99e None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:57.177532 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a8ebc-bfba-46f5-a145-225df1b8b99e None admin] PATH_INFO: `/v3/services/network` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:57.178195 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732a8ebc-bfba-46f5-a145-225df1b8b99e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:57.179615 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732a8ebc-bfba-46f5-a145-225df1b8b99e None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:57.189869 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-732a8ebc-bfba-46f5-a145-225df1b8b99e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 12:38:57.189869 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:57.189869 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 12:38:57.189869 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 12:38:57.189869 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 12:38:57.189869 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 12:38:57.189869 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 12:38:57.189869 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:38:57.189869 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 12:38:57.189869 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:57.189869 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 12:38:57.189869 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:57.189869 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:57.189869 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:57.189869 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:57.189869 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:57.189869 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:57.191096 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:57.191096 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:57.191096 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:57.191096 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:57.191096 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:57.191096 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:57.191096 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 12:38:57.191096 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 12:38:57.191096 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 12:38:57.191096 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 12:38:57.191096 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:57.191096 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:57.191096 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:57.191096 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:57.191096 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:57.191096 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:57.192106 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:57.192106 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:57.192106 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:57.192106 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:57.192106 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:57.192106 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:57.192106 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:57.192106 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:57.192106 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:57.192106 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:57.192106 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 12:38:57.192106 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:38:57.192106 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 12:38:57.192106 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:57.192844 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 416/818] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:38:57 2022] GET /identity/v3/services/network => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:57.194193 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a8ebc-bfba-46f5-a145-225df1b8b99e None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:57.206306 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f9965-fc14-4e0e-9b90-736ab17dc8c2 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:57.245394 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f0fa82-eae3-4896-b367-03375a1bad6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:57.246021 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0fa82-eae3-4896-b367-03375a1bad6f None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:57.246310 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0fa82-eae3-4896-b367-03375a1bad6f None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:57.246588 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0fa82-eae3-4896-b367-03375a1bad6f None admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:57.247506 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f0fa82-eae3-4896-b367-03375a1bad6f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:57.247958 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f0fa82-eae3-4896-b367-03375a1bad6f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:57.249392 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f0fa82-eae3-4896-b367-03375a1bad6f None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:57.268279 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 403/819] 10.222.0.20 () {56 vars in 1149 bytes} [Tue Jul 26 12:38:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:57.269120 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f0fa82-eae3-4896-b367-03375a1bad6f None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:57.311938 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a8ebc-bfba-46f5-a145-225df1b8b99e None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:57.344801 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:57.352577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47052b6d-e6ed-48a8-b3be-2c64ae423b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:57.353203 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47052b6d-e6ed-48a8-b3be-2c64ae423b5b None admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:57.353481 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47052b6d-e6ed-48a8-b3be-2c64ae423b5b None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:57.353749 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47052b6d-e6ed-48a8-b3be-2c64ae423b5b None admin] PATH_INFO: `/v3/endpoints` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:57.354457 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47052b6d-e6ed-48a8-b3be-2c64ae423b5b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:57.355873 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47052b6d-e6ed-48a8-b3be-2c64ae423b5b None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:57.381234 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 417/820] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:38:57 2022] POST /identity/v3/endpoints => generated 346 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:38:57.382129 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47052b6d-e6ed-48a8-b3be-2c64ae423b5b None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:57.404963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f0fa82-eae3-4896-b367-03375a1bad6f None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:57.421187 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:57.474908 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0134c6-8aae-40a7-bc7a-7dffe3ea7ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:57.476166 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0134c6-8aae-40a7-bc7a-7dffe3ea7ed6 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:57.476555 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0134c6-8aae-40a7-bc7a-7dffe3ea7ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:57.476768 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0134c6-8aae-40a7-bc7a-7dffe3ea7ed6 None admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:57.477447 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0134c6-8aae-40a7-bc7a-7dffe3ea7ed6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:57.477752 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0134c6-8aae-40a7-bc7a-7dffe3ea7ed6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:57.480091 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0134c6-8aae-40a7-bc7a-7dffe3ea7ed6 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:57.497979 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 404/821] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:57.499915 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0134c6-8aae-40a7-bc7a-7dffe3ea7ed6 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:57.509118 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47052b6d-e6ed-48a8-b3be-2c64ae423b5b None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:57.552899 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114993f-6298-49aa-b9cc-347cd75cc2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:57.553823 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114993f-6298-49aa-b9cc-347cd75cc2b0 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:57.554702 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114993f-6298-49aa-b9cc-347cd75cc2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:57.554702 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114993f-6298-49aa-b9cc-347cd75cc2b0 None admin] PATH_INFO: `/v3/services` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:57.555042 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7114993f-6298-49aa-b9cc-347cd75cc2b0 None admin] RBAC: Adding query filter params (name=network) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:57.555345 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7114993f-6298-49aa-b9cc-347cd75cc2b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:57.557430 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7114993f-6298-49aa-b9cc-347cd75cc2b0 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:57.561959 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 418/822] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Jul 26 12:38:57 2022] GET /identity/v3/services?name=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:57.562772 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114993f-6298-49aa-b9cc-347cd75cc2b0 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:57.649838 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0134c6-8aae-40a7-bc7a-7dffe3ea7ed6 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:57.677278 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:57.681573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:57.689614 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114993f-6298-49aa-b9cc-347cd75cc2b0 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:57.697469 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a1c87a-55e9-4a86-9551-a460ccb04d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:57.698136 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a1c87a-55e9-4a86-9551-a460ccb04d2b None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:57.698949 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a1c87a-55e9-4a86-9551-a460ccb04d2b None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:57.699229 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a1c87a-55e9-4a86-9551-a460ccb04d2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:57.713760 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a1c87a-55e9-4a86-9551-a460ccb04d2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:57.715555 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a1c87a-55e9-4a86-9551-a460ccb04d2b None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:57.729199 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b587770-bf34-45ac-bdf6-c91291432358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:57.730143 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b587770-bf34-45ac-bdf6-c91291432358 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:57.730631 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b587770-bf34-45ac-bdf6-c91291432358 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:57.731124 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b587770-bf34-45ac-bdf6-c91291432358 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:57.734675 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-01a1c87a-55e9-4a86-9551-a460ccb04d2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:57.734675 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:57.734675 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:57.734675 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:57.734675 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:57.734675 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:57.734675 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:57.734675 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:57.734675 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:57.734675 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:57.734675 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:57.734675 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:57.734675 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:57.734675 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:57.734675 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:57.735552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:57.735552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:57.735552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:57.735552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:57.735552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:57.735552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:57.735552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:57.735552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:57.735552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:57.735552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:57.735552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:57.735552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:57.735552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:57.735552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:57.735552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:57.735552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:57.735552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:57.736498 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:57.736498 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:57.736498 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:57.736498 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:57.736498 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 405/823] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:57.736809 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a1c87a-55e9-4a86-9551-a460ccb04d2b None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:57.746441 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b587770-bf34-45ac-bdf6-c91291432358 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:57.748313 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b587770-bf34-45ac-bdf6-c91291432358 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:57.761039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-5b587770-bf34-45ac-bdf6-c91291432358 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:57.761039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:57.761039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:57.761039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:57.761039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:57.761039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:57.761039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:57.761039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:57.761039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:57.761039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:57.761039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:57.761039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:57.761039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 12:38:57.761039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 12:38:57.761039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 12:38:57.762132 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 12:38:57.762132 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:57.762132 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:57.762132 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:57.762132 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:57.762132 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:57.762132 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:57.762132 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:57.762132 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:57.762132 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:57.762132 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:57.762132 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:57.762132 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:57.762132 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:57.762132 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:57.762132 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:57.762132 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:57.763196 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 12:38:57.763196 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 12:38:57.763196 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 12:38:57.763196 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 12:38:57.763196 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 12:38:57.763196 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 12:38:57.763196 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:57.763196 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:57.769459 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 419/824] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:38:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:57.769459 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b587770-bf34-45ac-bdf6-c91291432358 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:57.877391 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a1c87a-55e9-4a86-9551-a460ccb04d2b None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:57.878077 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b587770-bf34-45ac-bdf6-c91291432358 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:57.921192 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5784f4a1-70c7-4268-b32c-8054da7e18c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:57.922006 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5784f4a1-70c7-4268-b32c-8054da7e18c8 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:57.922648 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5784f4a1-70c7-4268-b32c-8054da7e18c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:57.922648 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5784f4a1-70c7-4268-b32c-8054da7e18c8 None admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:57.923620 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5784f4a1-70c7-4268-b32c-8054da7e18c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:57.924229 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5784f4a1-70c7-4268-b32c-8054da7e18c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:57.925850 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5784f4a1-70c7-4268-b32c-8054da7e18c8 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:57.938909 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2c7e4e-0099-41f4-b3b0-523e028363d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:57.940059 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c7e4e-0099-41f4-b3b0-523e028363d8 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:57.940506 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c7e4e-0099-41f4-b3b0-523e028363d8 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:57.940921 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c7e4e-0099-41f4-b3b0-523e028363d8 None admin] PATH_INFO: `/v3/services` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:57.941825 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2c7e4e-0099-41f4-b3b0-523e028363d8 None admin] RBAC: Adding query filter params (type=network) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:57.942366 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2c7e4e-0099-41f4-b3b0-523e028363d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:57.945592 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2c7e4e-0099-41f4-b3b0-523e028363d8 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:57.946268 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 420/825] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:57.947202 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5784f4a1-70c7-4268-b32c-8054da7e18c8 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:57.958132 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 406/826] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Jul 26 12:38:57 2022] GET /identity/v3/services?type=network => generated 350 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:57.959506 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2c7e4e-0099-41f4-b3b0-523e028363d8 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:57.966756 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:58.001901 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:58.060465 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5784f4a1-70c7-4268-b32c-8054da7e18c8 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:58.078098 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2c7e4e-0099-41f4-b3b0-523e028363d8 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:58.097713 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617299cd-ef7c-413f-b280-400e4b384202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:58.098112 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617299cd-ef7c-413f-b280-400e4b384202 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:58.098395 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617299cd-ef7c-413f-b280-400e4b384202 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:58.098668 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617299cd-ef7c-413f-b280-400e4b384202 None admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:58.099477 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617299cd-ef7c-413f-b280-400e4b384202 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:58.099898 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617299cd-ef7c-413f-b280-400e4b384202 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:58.101345 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617299cd-ef7c-413f-b280-400e4b384202 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:58.115800 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 421/827] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 26 12:38:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:58.116909 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617299cd-ef7c-413f-b280-400e4b384202 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:58.122404 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a7ad1-d74c-41e5-8826-b388acad8bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:58.123398 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a7ad1-d74c-41e5-8826-b388acad8bd6 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:58.123906 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a7ad1-d74c-41e5-8826-b388acad8bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:58.124391 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a7ad1-d74c-41e5-8826-b388acad8bd6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:58.132308 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a7ad1-d74c-41e5-8826-b388acad8bd6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:58.134127 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a7ad1-d74c-41e5-8826-b388acad8bd6 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:58.147415 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-023a7ad1-d74c-41e5-8826-b388acad8bd6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:58.147415 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:58.147415 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:58.147415 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:58.147415 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:58.147415 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:58.147415 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:58.147415 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:58.147415 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:58.147415 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:58.147415 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:58.147415 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:58.147415 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 12:38:58.147415 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 12:38:58.147415 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 12:38:58.148340 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 12:38:58.148340 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:58.148340 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:58.148340 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:58.148340 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:58.148340 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:58.148340 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:58.148340 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:58.148340 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:58.148340 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:58.148340 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:58.148340 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:58.148340 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:58.148340 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:58.148340 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:58.148340 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:58.148340 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:58.149211 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 12:38:58.149211 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 12:38:58.149211 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 12:38:58.149211 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 12:38:58.149211 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 12:38:58.149211 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 12:38:58.149211 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:38:58.149211 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:38:58.151666 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 407/828] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:38:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:58.152702 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a7ad1-d74c-41e5-8826-b388acad8bd6 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:58.212139 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:58.238513 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617299cd-ef7c-413f-b280-400e4b384202 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:58.259239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:58.267278 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a7ad1-d74c-41e5-8826-b388acad8bd6 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:58.278912 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed2d99-b575-4998-bf80-8d1da601e62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:58.280362 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed2d99-b575-4998-bf80-8d1da601e62a None admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:58.281082 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed2d99-b575-4998-bf80-8d1da601e62a None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:58.281812 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed2d99-b575-4998-bf80-8d1da601e62a None admin] PATH_INFO: `/v3/endpoints` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:58.283102 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ed2d99-b575-4998-bf80-8d1da601e62a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:58.284714 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ed2d99-b575-4998-bf80-8d1da601e62a None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:58.309770 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 422/829] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:38:58 2022] POST /identity/v3/endpoints => generated 327 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:38:58.311164 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed2d99-b575-4998-bf80-8d1da601e62a None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:58.332182 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e28be42-843b-453a-aa57-d27c28bdaeec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:58.332836 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e28be42-843b-453a-aa57-d27c28bdaeec None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:58.333038 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e28be42-843b-453a-aa57-d27c28bdaeec None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:58.333495 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e28be42-843b-453a-aa57-d27c28bdaeec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:58.334733 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e28be42-843b-453a-aa57-d27c28bdaeec None admin] RBAC: Adding query filter params (role.id=0e78444e43c744668178971af4087ca0, scope.project.id=644278e7e8934b37a1e7ab20d35764cd, user.id=0f5d6f3c12244c0faf26048513dc49e7) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:58.335221 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e28be42-843b-453a-aa57-d27c28bdaeec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:58.337116 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e28be42-843b-453a-aa57-d27c28bdaeec None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:58.359665 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 408/830] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:38:58 2022] GET /identity/v3/role_assignments?user.id=0f5d6f3c12244c0faf26048513dc49e7&scope.project.id=644278e7e8934b37a1e7ab20d35764cd&role.id=0e78444e43c744668178971af4087ca0 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:58.479480 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed2d99-b575-4998-bf80-8d1da601e62a None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:58.513864 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:58.520176 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cdfe06-5299-4f2a-9c07-c3131660f601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:58.522213 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cdfe06-5299-4f2a-9c07-c3131660f601 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:58.522734 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cdfe06-5299-4f2a-9c07-c3131660f601 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:58.523215 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cdfe06-5299-4f2a-9c07-c3131660f601 None admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:58.524270 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cdfe06-5299-4f2a-9c07-c3131660f601 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:58.524800 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cdfe06-5299-4f2a-9c07-c3131660f601 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:58.526582 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cdfe06-5299-4f2a-9c07-c3131660f601 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:58.541777 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 423/831] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 26 12:38:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:58.548288 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e28be42-843b-453a-aa57-d27c28bdaeec None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:58.675285 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e28be42-843b-453a-aa57-d27c28bdaeec None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:58.713852 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c76cb8-ca8f-45b2-b6f1-f03c964caf32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:58.714829 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c76cb8-ca8f-45b2-b6f1-f03c964caf32 None admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:58.715313 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c76cb8-ca8f-45b2-b6f1-f03c964caf32 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:58.715872 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c76cb8-ca8f-45b2-b6f1-f03c964caf32 None admin] PATH_INFO: `/v3/projects/644278e7e8934b37a1e7ab20d35764cd/users/30fc532f464843a3ad35e4d3c387bb3a/roles/9c68629183974be9bfb1ed95f69912ee` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:58.733271 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c76cb8-ca8f-45b2-b6f1-f03c964caf32 None admin] RBAC: Authorizing `identity:create_grant(project_id=644278e7e8934b37a1e7ab20d35764cd, user_id=30fc532f464843a3ad35e4d3c387bb3a, role_id=9c68629183974be9bfb1ed95f69912ee)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:58.735071 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c76cb8-ca8f-45b2-b6f1-f03c964caf32 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:58.753299 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 409/832] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 26 12:38:58 2022] PUT /identity/v3/projects/644278e7e8934b37a1e7ab20d35764cd/users/30fc532f464843a3ad35e4d3c387bb3a/roles/9c68629183974be9bfb1ed95f69912ee => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:38:59.001280 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf206a8-f8d7-4162-a7bb-b063b8a25266 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:59.001801 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf206a8-f8d7-4162-a7bb-b063b8a25266 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:59.001801 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf206a8-f8d7-4162-a7bb-b063b8a25266 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:59.003332 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 424/833] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:38:59.008821 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5deb0a-5d57-4231-9f7e-35ef9f40a951 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:59.009614 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5deb0a-5d57-4231-9f7e-35ef9f40a951 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:59.010085 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5deb0a-5d57-4231-9f7e-35ef9f40a951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:59.023267 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:59.030936 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-fb5deb0a-5d57-4231-9f7e-35ef9f40a951 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:59.032281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-fb5deb0a-5d57-4231-9f7e-35ef9f40a951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:59.102420 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 410/834] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:38:59.125386 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf206a8-f8d7-4162-a7bb-b063b8a25266 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:59.240563 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf206a8-f8d7-4162-a7bb-b063b8a25266 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:59.278077 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f71bbf-deb7-4e1d-9260-97b0fad26f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:59.278944 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f71bbf-deb7-4e1d-9260-97b0fad26f67 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:59.279228 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f71bbf-deb7-4e1d-9260-97b0fad26f67 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:59.279506 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f71bbf-deb7-4e1d-9260-97b0fad26f67 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:59.291324 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f71bbf-deb7-4e1d-9260-97b0fad26f67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:38:59.296733 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-14f71bbf-deb7-4e1d-9260-97b0fad26f67 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:38:59.296733 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:59.296733 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:59.296733 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:59.296733 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:59.296733 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:59.296733 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:59.296733 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:59.296733 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:59.296733 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:59.296733 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:59.296733 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:59.296733 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:38:59.296733 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:38:59.296733 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:38:59.296733 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:38:59.297633 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:38:59.297633 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:38:59.297633 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:38:59.297633 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:38:59.297633 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:59.297633 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:59.297633 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:59.297633 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:59.297633 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:59.297633 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:59.297633 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:59.297633 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:59.297633 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:59.297633 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:59.297633 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:59.297633 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:59.298484 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:59.298484 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:59.298484 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:59.298484 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:59.298484 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:38:59.298484 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:38:59.298484 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:38:59.298484 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:38:59.298484 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:38:59.298484 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:38:59.298484 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:38:59.298484 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:59.298484 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 425/835] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 26 12:38:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:59.304673 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5deb0a-5d57-4231-9f7e-35ef9f40a951 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:59.404124 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5deb0a-5d57-4231-9f7e-35ef9f40a951 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:59.439843 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4315ed32-c4a4-482b-ad2c-b6576e8f397f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:59.440753 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4315ed32-c4a4-482b-ad2c-b6576e8f397f None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:59.441206 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4315ed32-c4a4-482b-ad2c-b6576e8f397f None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:59.441648 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4315ed32-c4a4-482b-ad2c-b6576e8f397f None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:59.442555 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4315ed32-c4a4-482b-ad2c-b6576e8f397f None admin] RBAC: Adding query filter params (name=admin) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:59.443061 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4315ed32-c4a4-482b-ad2c-b6576e8f397f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:59.444643 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4315ed32-c4a4-482b-ad2c-b6576e8f397f None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:59.447052 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:59.453674 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 411/836] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:38:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:59.459282 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f71bbf-deb7-4e1d-9260-97b0fad26f67 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:59.516145 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:59.526823 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84bcbc4-d482-4fe6-a76e-e3966d9eddc7 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:59.527110 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84bcbc4-d482-4fe6-a76e-e3966d9eddc7 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:59.527373 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84bcbc4-d482-4fe6-a76e-e3966d9eddc7 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:59.529059 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 412/837] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:38:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 12:38:59.533896 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e042965a-b11a-40a4-9ca7-094ba8c7ca91 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:59.534175 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e042965a-b11a-40a4-9ca7-094ba8c7ca91 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:59.534444 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e042965a-b11a-40a4-9ca7-094ba8c7ca91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:59.553301 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-e042965a-b11a-40a4-9ca7-094ba8c7ca91 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:38:59.554485 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-e042965a-b11a-40a4-9ca7-094ba8c7ca91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:38:59.567259 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f71bbf-deb7-4e1d-9260-97b0fad26f67 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:59.605513 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9734405-b34a-4f0b-843e-e90e90aadcce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:59.606184 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9734405-b34a-4f0b-843e-e90e90aadcce None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:59.606573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9734405-b34a-4f0b-843e-e90e90aadcce None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:59.606853 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9734405-b34a-4f0b-843e-e90e90aadcce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:59.618821 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9734405-b34a-4f0b-843e-e90e90aadcce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:59.620537 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9734405-b34a-4f0b-843e-e90e90aadcce None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:59.631966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 413/838] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:38:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 12:38:59.636116 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-b9734405-b34a-4f0b-843e-e90e90aadcce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:59.636116 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:59.636116 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:59.636116 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:59.636116 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:59.636116 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:59.636116 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:59.636116 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:59.636116 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:59.636116 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:59.636116 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:59.636116 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:59.636116 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:38:59.636116 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:38:59.636116 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:38:59.636967 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:38:59.636967 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:59.636967 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:59.636967 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:59.636967 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:59.636967 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:59.636967 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:59.636967 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:59.636967 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:59.636967 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:59.636967 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:59.636967 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:59.636967 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:59.636967 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:59.636967 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:59.636967 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:59.636967 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:59.638238 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:38:59.638238 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:38:59.638238 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:38:59.638238 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:59.638238 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 426/839] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:38:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:59.644289 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e042965a-b11a-40a4-9ca7-094ba8c7ca91 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:59.648295 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9734405-b34a-4f0b-843e-e90e90aadcce None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:59.755078 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e042965a-b11a-40a4-9ca7-094ba8c7ca91 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:59.774665 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9734405-b34a-4f0b-843e-e90e90aadcce None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:59.777818 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:59.786875 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:38:59.800416 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a7aa5-7081-4a77-ae3c-1c4f34897101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:59.801050 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a7aa5-7081-4a77-ae3c-1c4f34897101 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:59.801337 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a7aa5-7081-4a77-ae3c-1c4f34897101 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:59.801613 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a7aa5-7081-4a77-ae3c-1c4f34897101 None admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:59.802838 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a7aa5-7081-4a77-ae3c-1c4f34897101 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:38:59.803146 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a7aa5-7081-4a77-ae3c-1c4f34897101 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:59.804857 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a7aa5-7081-4a77-ae3c-1c4f34897101 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:59.817548 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 414/840] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:38:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:38:59.823438 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a7aa5-7081-4a77-ae3c-1c4f34897101 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:59.831820 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7b0a14-5a5e-41f4-a859-7a2eae4a1b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:59.832474 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b0a14-5a5e-41f4-a859-7a2eae4a1b19 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:59.832750 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b0a14-5a5e-41f4-a859-7a2eae4a1b19 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:59.833017 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b0a14-5a5e-41f4-a859-7a2eae4a1b19 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:38:59.833695 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7b0a14-5a5e-41f4-a859-7a2eae4a1b19 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:38:59.835235 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7b0a14-5a5e-41f4-a859-7a2eae4a1b19 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:38:59.845852 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-9e7b0a14-5a5e-41f4-a859-7a2eae4a1b19 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 12:38:59.845852 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:59.845852 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 12:38:59.845852 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 12:38:59.845852 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 12:38:59.845852 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 12:38:59.845852 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 12:38:59.845852 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:38:59.845852 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 12:38:59.845852 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:59.845852 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 12:38:59.845852 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:59.845852 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:38:59.845852 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:38:59.845852 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:38:59.845852 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:38:59.845852 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:38:59.846937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:38:59.846937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:38:59.846937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:38:59.846937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:38:59.846937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:38:59.846937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:38:59.846937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 12:38:59.846937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 12:38:59.846937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 12:38:59.846937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 12:38:59.846937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:38:59.846937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:38:59.846937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:38:59.846937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:38:59.846937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:38:59.846937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:38:59.847819 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:38:59.847819 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:38:59.847819 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:38:59.847819 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:38:59.847819 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:38:59.847819 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:38:59.847819 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:38:59.847819 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:38:59.847819 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:38:59.847819 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:38:59.847819 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 12:38:59.847819 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:38:59.847819 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 12:38:59.847819 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:38:59.852080 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 427/841] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 26 12:38:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:38:59.853476 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7b0a14-5a5e-41f4-a859-7a2eae4a1b19 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:38:59.958388 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a7aa5-7081-4a77-ae3c-1c4f34897101 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:59.965110 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7b0a14-5a5e-41f4-a859-7a2eae4a1b19 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:38:59.996294 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e5e91d-8f07-4fc8-9e1b-d88c7ad6cccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:38:59.996959 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5e91d-8f07-4fc8-9e1b-d88c7ad6cccd None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:38:59.997240 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5e91d-8f07-4fc8-9e1b-d88c7ad6cccd None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:38:59.997514 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5e91d-8f07-4fc8-9e1b-d88c7ad6cccd None admin] PATH_INFO: `/v3/users/swift` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:00.006685 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e5e91d-8f07-4fc8-9e1b-d88c7ad6cccd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:00.008217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e5e91d-8f07-4fc8-9e1b-d88c7ad6cccd None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:00.014174 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321d4b06-e197-455a-826e-195d6c630d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:00.015460 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d4b06-e197-455a-826e-195d6c630d33 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:00.015982 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d4b06-e197-455a-826e-195d6c630d33 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:00.016459 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d4b06-e197-455a-826e-195d6c630d33 None admin] PATH_INFO: `/v3/services` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:00.017382 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321d4b06-e197-455a-826e-195d6c630d33 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:00.017890 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321d4b06-e197-455a-826e-195d6c630d33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:00.019461 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321d4b06-e197-455a-826e-195d6c630d33 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:00.020876 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-61e5e91d-8f07-4fc8-9e1b-d88c7ad6cccd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 12:39:00.020876 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:00.020876 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:00.020876 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:00.020876 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:00.020876 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:00.020876 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:00.020876 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:00.020876 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:00.020876 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:00.020876 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:00.020876 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:00.020876 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 12:39:00.020876 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 12:39:00.020876 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 12:39:00.020876 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 12:39:00.022650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:00.022650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:00.022650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 12:39:00.022650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:39:00.022650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 12:39:00.022650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:39:00.022650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:00.022650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:00.022650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:00.022650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:00.022650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:00.022650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:00.022650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:00.022650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:00.022650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:00.022650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:00.022650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:00.022650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:00.023593 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:00.023593 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:00.023593 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 12:39:00.023593 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 12:39:00.023593 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 12:39:00.023593 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 12:39:00.023593 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 12:39:00.023593 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 12:39:00.023593 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 12:39:00.023593 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:00.023593 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 415/842] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 26 12:38:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:39:00.028331 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e5e91d-8f07-4fc8-9e1b-d88c7ad6cccd None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:00.029305 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 428/843] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 26 12:38:59 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:39:00.035356 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321d4b06-e197-455a-826e-195d6c630d33 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:00.074847 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:00.080888 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:00.134885 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e5e91d-8f07-4fc8-9e1b-d88c7ad6cccd None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:00.152356 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321d4b06-e197-455a-826e-195d6c630d33 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:00.169971 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb3934e-447c-4b71-9b4c-09cfe8be62d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:00.170597 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb3934e-447c-4b71-9b4c-09cfe8be62d8 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:00.170875 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb3934e-447c-4b71-9b4c-09cfe8be62d8 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:00.171165 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb3934e-447c-4b71-9b4c-09cfe8be62d8 None admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:00.171965 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb3934e-447c-4b71-9b4c-09cfe8be62d8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:00.172270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb3934e-447c-4b71-9b4c-09cfe8be62d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:00.173689 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb3934e-447c-4b71-9b4c-09cfe8be62d8 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:00.192257 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 416/844] 10.222.0.20 () {56 vars in 1135 bytes} [Tue Jul 26 12:39:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:39:00.194818 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b04edfa-e947-4f31-93d5-1f907fcdf3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:00.195441 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04edfa-e947-4f31-93d5-1f907fcdf3a4 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:00.199737 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04edfa-e947-4f31-93d5-1f907fcdf3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:00.199737 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04edfa-e947-4f31-93d5-1f907fcdf3a4 None admin] PATH_INFO: `/v3/services` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:00.199737 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b04edfa-e947-4f31-93d5-1f907fcdf3a4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:00.199737 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b04edfa-e947-4f31-93d5-1f907fcdf3a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:00.200072 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb3934e-447c-4b71-9b4c-09cfe8be62d8 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:00.200145 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b04edfa-e947-4f31-93d5-1f907fcdf3a4 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:00.207796 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 429/845] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 26 12:39:00 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:00.311508 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb3934e-447c-4b71-9b4c-09cfe8be62d8 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:00.320693 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:00.351900 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8970fa4-e4b9-4d76-896f-bd7bdffcaf2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:00.352445 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8970fa4-e4b9-4d76-896f-bd7bdffcaf2a None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:00.352725 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8970fa4-e4b9-4d76-896f-bd7bdffcaf2a None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:00.352993 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8970fa4-e4b9-4d76-896f-bd7bdffcaf2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:00.428289 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8970fa4-e4b9-4d76-896f-bd7bdffcaf2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:00.430038 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8970fa4-e4b9-4d76-896f-bd7bdffcaf2a None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:00.538043 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 417/846] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:39:00 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 12:39:00.543576 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b04edfa-e947-4f31-93d5-1f907fcdf3a4 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:00.645183 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:00.652165 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b04edfa-e947-4f31-93d5-1f907fcdf3a4 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:00.688372 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0efab9b-0266-4f94-a4ee-f59592e72693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:00.689317 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0efab9b-0266-4f94-a4ee-f59592e72693 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:00.689781 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0efab9b-0266-4f94-a4ee-f59592e72693 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:00.690256 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0efab9b-0266-4f94-a4ee-f59592e72693 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:00.702393 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0efab9b-0266-4f94-a4ee-f59592e72693 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:00.704284 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0efab9b-0266-4f94-a4ee-f59592e72693 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:00.719624 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-c0efab9b-0266-4f94-a4ee-f59592e72693 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:39:00.719624 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:00.719624 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:00.719624 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:00.719624 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:00.719624 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:00.719624 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:00.719624 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:00.719624 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:00.719624 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:00.719624 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:00.719624 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:00.719624 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:39:00.719624 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:39:00.719624 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:39:00.720577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:39:00.720577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:00.720577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:00.720577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:00.720577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:00.720577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:00.720577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:00.720577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:00.720577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:00.720577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:00.720577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:00.720577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:00.720577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:00.720577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:00.720577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:00.720577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:00.720577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:00.721465 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:39:00.721465 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:39:00.721465 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:39:00.721465 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:00.723393 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 430/847] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:39:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:39:00.730888 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8970fa4-e4b9-4d76-896f-bd7bdffcaf2a None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:00.741845 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:00.787450 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a0f9fb-9cf8-47b4-812e-b9ead20c099a None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:00.787803 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a0f9fb-9cf8-47b4-812e-b9ead20c099a None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:00.788089 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a0f9fb-9cf8-47b4-812e-b9ead20c099a None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:00.789720 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 431/848] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:39:00.794852 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517f110-e61a-47bf-9e19-5da49f3de9b2 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:00.795139 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517f110-e61a-47bf-9e19-5da49f3de9b2 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:00.795417 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517f110-e61a-47bf-9e19-5da49f3de9b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:00.814240 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-d517f110-e61a-47bf-9e19-5da49f3de9b2 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:00.815550 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-d517f110-e61a-47bf-9e19-5da49f3de9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:00.838091 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8970fa4-e4b9-4d76-896f-bd7bdffcaf2a None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:00.876200 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5935dc1-b258-4f4d-bda8-8aaf3dbfa1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:00.877030 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5935dc1-b258-4f4d-bda8-8aaf3dbfa1b1 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:00.877310 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5935dc1-b258-4f4d-bda8-8aaf3dbfa1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:00.877670 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5935dc1-b258-4f4d-bda8-8aaf3dbfa1b1 None admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:00.878527 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5935dc1-b258-4f4d-bda8-8aaf3dbfa1b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:00.878836 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5935dc1-b258-4f4d-bda8-8aaf3dbfa1b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:00.880242 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5935dc1-b258-4f4d-bda8-8aaf3dbfa1b1 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:00.898309 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 418/849] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:39:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:00.899682 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5935dc1-b258-4f4d-bda8-8aaf3dbfa1b1 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:00.906671 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 432/850] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:00 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 12:39:00.928591 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d517f110-e61a-47bf-9e19-5da49f3de9b2 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:00.975450 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:01.011271 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5935dc1-b258-4f4d-bda8-8aaf3dbfa1b1 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:01.024399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:01.047553 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d517f110-e61a-47bf-9e19-5da49f3de9b2 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:01.048080 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bdf323-efdb-43cd-9b10-88314b929b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:01.048781 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bdf323-efdb-43cd-9b10-88314b929b66 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:01.049216 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bdf323-efdb-43cd-9b10-88314b929b66 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:01.049537 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bdf323-efdb-43cd-9b10-88314b929b66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:01.058418 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bdf323-efdb-43cd-9b10-88314b929b66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:01.060202 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bdf323-efdb-43cd-9b10-88314b929b66 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:01.071514 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-c2bdf323-efdb-43cd-9b10-88314b929b66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:39:01.071514 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:01.071514 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:01.071514 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:01.071514 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:01.071514 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:01.071514 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:01.071514 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:01.071514 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:01.071514 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:01.071514 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:01.071514 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:01.071514 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 12:39:01.071514 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 12:39:01.071514 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 12:39:01.072570 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 12:39:01.072570 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:01.072570 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:01.072570 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:01.072570 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:01.072570 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:01.072570 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:01.072570 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:01.072570 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:01.072570 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:01.072570 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:01.072570 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:01.072570 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:01.072570 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:01.072570 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:01.072570 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:01.072570 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:01.073437 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 12:39:01.073437 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 12:39:01.073437 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 12:39:01.073437 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 12:39:01.073437 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 12:39:01.073437 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 12:39:01.073437 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:39:01.073437 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:01.077601 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 419/851] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:39:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:39:01.078711 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27fe5d9-c0da-418f-8883-6aa7c0985e77 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:01.078986 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27fe5d9-c0da-418f-8883-6aa7c0985e77 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:01.079267 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27fe5d9-c0da-418f-8883-6aa7c0985e77 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:01.080809 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 420/852] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:39:01.081502 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27fe5d9-c0da-418f-8883-6aa7c0985e77 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:01.089692 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d0445-6af9-4c7d-b454-4cf36456d449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:01.090335 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d0445-6af9-4c7d-b454-4cf36456d449 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:01.090636 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d0445-6af9-4c7d-b454-4cf36456d449 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:01.090873 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d0445-6af9-4c7d-b454-4cf36456d449 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:01.091491 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93d0445-6af9-4c7d-b454-4cf36456d449 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:01.093341 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93d0445-6af9-4c7d-b454-4cf36456d449 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:01.103614 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-d93d0445-6af9-4c7d-b454-4cf36456d449 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 12:39:01.103614 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:01.103614 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 12:39:01.103614 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 12:39:01.103614 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 12:39:01.103614 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 12:39:01.103614 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 12:39:01.103614 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:39:01.103614 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 12:39:01.103614 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:01.103614 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 12:39:01.103614 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:01.103614 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:01.103614 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:01.103614 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:01.103614 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:01.104588 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:01.104588 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:01.104588 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:01.104588 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:01.104588 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:01.104588 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:01.104588 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:01.104588 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 12:39:01.104588 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 12:39:01.104588 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 12:39:01.104588 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 12:39:01.104588 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:01.104588 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:01.104588 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:01.104588 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:01.104588 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:01.105423 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:01.105423 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:01.105423 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:01.105423 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:01.105423 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:01.105423 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:01.105423 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:01.105423 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:01.105423 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:01.105423 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:01.105423 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:01.105423 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 12:39:01.105423 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:39:01.105423 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 12:39:01.105423 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:01.105423 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 433/853] 10.222.0.20 () {56 vars in 1132 bytes} [Tue Jul 26 12:39:00 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:39:01.106290 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4446a-dd93-4c4b-bf41-f95b93630a43 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:01.107320 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4446a-dd93-4c4b-bf41-f95b93630a43 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:01.107320 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4446a-dd93-4c4b-bf41-f95b93630a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:01.124600 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-f5a4446a-dd93-4c4b-bf41-f95b93630a43 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:01.125790 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-f5a4446a-dd93-4c4b-bf41-f95b93630a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:01.201811 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27fe5d9-c0da-418f-8883-6aa7c0985e77 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:01.203687 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 434/854] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:01 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:39:01.203687 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a4446a-dd93-4c4b-bf41-f95b93630a43 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:01.234422 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:01.244473 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32680ffd-1390-43ee-a4f8-e63a0fb90993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:01.245188 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32680ffd-1390-43ee-a4f8-e63a0fb90993 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:01.245501 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32680ffd-1390-43ee-a4f8-e63a0fb90993 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:01.245781 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32680ffd-1390-43ee-a4f8-e63a0fb90993 None admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:01.246519 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32680ffd-1390-43ee-a4f8-e63a0fb90993 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:01.246823 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32680ffd-1390-43ee-a4f8-e63a0fb90993 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:01.248276 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32680ffd-1390-43ee-a4f8-e63a0fb90993 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:01.265739 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 421/855] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 26 12:39:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:01.265739 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32680ffd-1390-43ee-a4f8-e63a0fb90993 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:01.293839 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:01.325289 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a4446a-dd93-4c4b-bf41-f95b93630a43 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:01.362452 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf7b70a-1450-4806-98d9-605c71486515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:01.363139 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7b70a-1450-4806-98d9-605c71486515 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:01.363350 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7b70a-1450-4806-98d9-605c71486515 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:01.363627 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7b70a-1450-4806-98d9-605c71486515 None admin] PATH_INFO: `/v3/services` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:01.364330 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7b70a-1450-4806-98d9-605c71486515 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:01.364636 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7b70a-1450-4806-98d9-605c71486515 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:01.366342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7b70a-1450-4806-98d9-605c71486515 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:01.368135 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32680ffd-1390-43ee-a4f8-e63a0fb90993 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:01.375808 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 435/856] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Jul 26 12:39:01 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:01.375808 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf7b70a-1450-4806-98d9-605c71486515 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:01.406349 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7437a9-ad84-4c77-8f62-e0cf8cba855a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:01.406973 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7437a9-ad84-4c77-8f62-e0cf8cba855a None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:01.407253 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7437a9-ad84-4c77-8f62-e0cf8cba855a None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:01.407539 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7437a9-ad84-4c77-8f62-e0cf8cba855a None admin] PATH_INFO: `/v3/roles/service` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:01.419881 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7437a9-ad84-4c77-8f62-e0cf8cba855a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:39:01.424806 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-ce7437a9-ad84-4c77-8f62-e0cf8cba855a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 12:39:01.424806 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:01.424806 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:01.424806 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:01.424806 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:01.424806 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:01.424806 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:01.424806 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:01.424806 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:01.424806 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:01.424806 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:01.424806 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:01.424806 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:39:01.424806 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:39:01.424806 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:39:01.424806 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:39:01.425731 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:39:01.425731 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:39:01.425731 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:39:01.425731 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:39:01.425731 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:01.425731 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:01.425731 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:01.425731 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:01.425731 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:01.425731 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:01.425731 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:01.425731 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:01.425731 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:01.425731 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:01.425731 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:01.425731 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:01.426577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:01.426577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:01.426577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:01.426577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:01.426577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:39:01.426577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:39:01.426577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:39:01.426577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:39:01.426577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:39:01.426577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:39:01.426577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 12:39:01.426577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:01.426577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 422/857] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 26 12:39:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:39:01.427253 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7437a9-ad84-4c77-8f62-e0cf8cba855a None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:01.477880 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:01.485833 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf7b70a-1450-4806-98d9-605c71486515 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:01.522123 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f439b389-79e1-4414-a79b-012ed88c985f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:01.523086 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f439b389-79e1-4414-a79b-012ed88c985f None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:01.523365 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f439b389-79e1-4414-a79b-012ed88c985f None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:01.523667 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f439b389-79e1-4414-a79b-012ed88c985f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:01.524464 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f439b389-79e1-4414-a79b-012ed88c985f None admin] RBAC: Adding query filter params (role.id=0e78444e43c744668178971af4087ca0, scope.project.id=644278e7e8934b37a1e7ab20d35764cd, user.id=cd2483f74d7d48b1a7b8ffe32f63abde) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:01.524770 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f439b389-79e1-4414-a79b-012ed88c985f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:01.526202 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f439b389-79e1-4414-a79b-012ed88c985f None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:01.529235 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:01.539270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7437a9-ad84-4c77-8f62-e0cf8cba855a None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:01.540030 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 436/858] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:39:01 2022] GET /identity/v3/role_assignments?user.id=cd2483f74d7d48b1a7b8ffe32f63abde&scope.project.id=644278e7e8934b37a1e7ab20d35764cd&role.id=0e78444e43c744668178971af4087ca0 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:01.541520 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f439b389-79e1-4414-a79b-012ed88c985f None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:01.579511 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac741d-510c-435d-a666-08126effb660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:01.580191 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac741d-510c-435d-a666-08126effb660 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:01.580556 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac741d-510c-435d-a666-08126effb660 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:01.580773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac741d-510c-435d-a666-08126effb660 None admin] PATH_INFO: `/v3/services` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:01.581425 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ac741d-510c-435d-a666-08126effb660 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:01.581726 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ac741d-510c-435d-a666-08126effb660 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:01.583142 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ac741d-510c-435d-a666-08126effb660 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:01.589418 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 423/859] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Jul 26 12:39:01 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:01.682710 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f439b389-79e1-4414-a79b-012ed88c985f None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:01.726103 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a050530e-335b-441a-924d-79b1efa215c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:01.727174 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a050530e-335b-441a-924d-79b1efa215c1 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:01.727720 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a050530e-335b-441a-924d-79b1efa215c1 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:01.728227 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a050530e-335b-441a-924d-79b1efa215c1 None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:01.729514 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a050530e-335b-441a-924d-79b1efa215c1 None admin] RBAC: Adding query filter params (name=service) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:01.729997 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a050530e-335b-441a-924d-79b1efa215c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:01.731449 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a050530e-335b-441a-924d-79b1efa215c1 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:01.741876 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 437/860] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Jul 26 12:39:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:01.747665 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac741d-510c-435d-a666-08126effb660 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:01.873324 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac741d-510c-435d-a666-08126effb660 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:01.885501 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:01.912731 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d018f555-9fb1-4649-a149-1d446408ad7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:01.913741 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018f555-9fb1-4649-a149-1d446408ad7f None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:01.914221 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018f555-9fb1-4649-a149-1d446408ad7f None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:01.914688 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018f555-9fb1-4649-a149-1d446408ad7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:01.925939 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d018f555-9fb1-4649-a149-1d446408ad7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:01.927868 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d018f555-9fb1-4649-a149-1d446408ad7f None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:01.943798 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-d018f555-9fb1-4649-a149-1d446408ad7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:39:01.943798 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:01.943798 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:01.943798 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:01.943798 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:01.943798 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:01.943798 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:01.943798 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:01.943798 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:01.943798 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:01.943798 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:01.943798 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:01.943798 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:39:01.943798 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:39:01.943798 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:39:01.944751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:39:01.944751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:01.944751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:01.944751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:01.944751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:01.944751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:01.944751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:01.944751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:01.944751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:01.944751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:01.944751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:01.944751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:01.944751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:01.944751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:01.944751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:01.944751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:01.944751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:01.945640 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:39:01.945640 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:39:01.945640 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:39:01.945640 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:01.951903 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 424/861] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:39:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:39:01.953746 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a050530e-335b-441a-924d-79b1efa215c1 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:02.018884 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:02.060888 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a050530e-335b-441a-924d-79b1efa215c1 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:02.097906 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfe6490-a966-4f70-a040-72de17b3d733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:02.098527 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe6490-a966-4f70-a040-72de17b3d733 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:02.098807 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe6490-a966-4f70-a040-72de17b3d733 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:02.099080 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe6490-a966-4f70-a040-72de17b3d733 None admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:02.099974 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfe6490-a966-4f70-a040-72de17b3d733 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:02.100281 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfe6490-a966-4f70-a040-72de17b3d733 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:02.101668 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfe6490-a966-4f70-a040-72de17b3d733 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:02.115766 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 438/862] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:39:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:02.121362 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d018f555-9fb1-4649-a149-1d446408ad7f None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:02.230889 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d018f555-9fb1-4649-a149-1d446408ad7f None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:02.267871 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc138fa-e2e4-4699-96eb-fd2dfce7c51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:02.268509 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc138fa-e2e4-4699-96eb-fd2dfce7c51d None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:02.268783 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc138fa-e2e4-4699-96eb-fd2dfce7c51d None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:02.269108 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc138fa-e2e4-4699-96eb-fd2dfce7c51d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:02.277141 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc138fa-e2e4-4699-96eb-fd2dfce7c51d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:02.278594 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc138fa-e2e4-4699-96eb-fd2dfce7c51d None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:02.290856 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-7bc138fa-e2e4-4699-96eb-fd2dfce7c51d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 12:39:02.290856 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:02.290856 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:02.290856 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:02.290856 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:02.290856 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:02.290856 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:02.290856 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:02.290856 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:02.290856 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:02.290856 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:02.290856 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:02.290856 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 12:39:02.290856 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 12:39:02.290856 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 12:39:02.290856 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 12:39:02.291895 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:02.291895 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:02.291895 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 12:39:02.291895 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:39:02.291895 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 12:39:02.291895 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:39:02.291895 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:02.291895 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:02.291895 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:02.291895 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:02.291895 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:02.291895 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:02.291895 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:02.291895 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:02.291895 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:02.291895 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:02.291895 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:02.291895 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:02.293475 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:02.293475 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:02.293475 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 12:39:02.293475 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 12:39:02.293475 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 12:39:02.293475 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 12:39:02.293475 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 12:39:02.293475 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 12:39:02.293475 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 12:39:02.293475 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:02.293475 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 425/863] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Jul 26 12:39:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:39:02.297967 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfe6490-a966-4f70-a040-72de17b3d733 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:02.344451 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d384a0-9f11-40c6-bf6b-87e36281435d None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:02.344780 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d384a0-9f11-40c6-bf6b-87e36281435d None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:02.345377 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d384a0-9f11-40c6-bf6b-87e36281435d None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:02.347343 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 426/864] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 12:39:02.353127 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b560cd-f502-45e4-bdd8-23472c5724fb None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:02.353758 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b560cd-f502-45e4-bdd8-23472c5724fb None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:02.354155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b560cd-f502-45e4-bdd8-23472c5724fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:02.379232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-19b560cd-f502-45e4-bdd8-23472c5724fb None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:02.381502 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-19b560cd-f502-45e4-bdd8-23472c5724fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:02.405266 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:02.414707 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfe6490-a966-4f70-a040-72de17b3d733 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:02.439598 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:02.453968 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d3e0e5-194a-4b3a-a3cd-000e03af62d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:02.454977 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d3e0e5-194a-4b3a-a3cd-000e03af62d4 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:02.455473 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d3e0e5-194a-4b3a-a3cd-000e03af62d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:02.456038 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d3e0e5-194a-4b3a-a3cd-000e03af62d4 None admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:02.457178 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d3e0e5-194a-4b3a-a3cd-000e03af62d4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:02.457687 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d3e0e5-194a-4b3a-a3cd-000e03af62d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:02.459459 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d3e0e5-194a-4b3a-a3cd-000e03af62d4 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:02.475680 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 427/865] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:02 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:39:02.488732 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 439/866] 10.222.0.20 () {56 vars in 1149 bytes} [Tue Jul 26 12:39:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:02.494918 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b560cd-f502-45e4-bdd8-23472c5724fb None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:02.500347 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d3e0e5-194a-4b3a-a3cd-000e03af62d4 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:02.599271 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b560cd-f502-45e4-bdd8-23472c5724fb None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:02.622562 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d3e0e5-194a-4b3a-a3cd-000e03af62d4 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:02.637498 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c2faa6-7383-488b-87b4-717c0dbcf027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:02.638213 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c2faa6-7383-488b-87b4-717c0dbcf027 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:02.638769 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c2faa6-7383-488b-87b4-717c0dbcf027 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:02.638905 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c2faa6-7383-488b-87b4-717c0dbcf027 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:02.660234 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8bf40b-ac06-4743-b2e5-1b89f036ae62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:02.661139 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8bf40b-ac06-4743-b2e5-1b89f036ae62 None admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:02.661647 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8bf40b-ac06-4743-b2e5-1b89f036ae62 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:02.662144 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8bf40b-ac06-4743-b2e5-1b89f036ae62 None admin] PATH_INFO: `/v3/services` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:02.662786 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:02.663245 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8bf40b-ac06-4743-b2e5-1b89f036ae62 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:02.665039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8bf40b-ac06-4743-b2e5-1b89f036ae62 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:02.683182 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 440/867] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Jul 26 12:39:02 2022] POST /identity/v3/services => generated 251 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:39:02.732016 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c2faa6-7383-488b-87b4-717c0dbcf027 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:02.733499 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c2faa6-7383-488b-87b4-717c0dbcf027 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:02.852996 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 428/868] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:39:02 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 12:39:02.858386 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8bf40b-ac06-4743-b2e5-1b89f036ae62 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:02.905063 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:02.973117 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8bf40b-ac06-4743-b2e5-1b89f036ae62 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:03.012814 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daefa1e-acac-40a5-a0fa-807152ba87d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:03.013465 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daefa1e-acac-40a5-a0fa-807152ba87d1 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:03.013749 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daefa1e-acac-40a5-a0fa-807152ba87d1 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:03.014022 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daefa1e-acac-40a5-a0fa-807152ba87d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:03.025122 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daefa1e-acac-40a5-a0fa-807152ba87d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:03.026582 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daefa1e-acac-40a5-a0fa-807152ba87d1 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:03.042015 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-4daefa1e-acac-40a5-a0fa-807152ba87d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:39:03.042015 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:03.042015 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:03.042015 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:03.042015 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:03.042015 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:03.042015 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:03.042015 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:03.042015 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:03.042015 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:03.042015 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:03.042015 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:03.042015 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:39:03.042015 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:39:03.042015 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:39:03.042925 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:39:03.042925 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:03.042925 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:03.042925 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:03.042925 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:03.042925 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:03.042925 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:03.042925 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:03.042925 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:03.042925 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:03.042925 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:03.042925 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:03.042925 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:03.042925 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:03.042925 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:03.042925 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:03.042925 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:03.043833 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:39:03.043833 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:39:03.043833 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:39:03.043833 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:03.043833 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 441/869] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:39:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:39:03.048840 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c2faa6-7383-488b-87b4-717c0dbcf027 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:03.103858 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:03.154408 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c2faa6-7383-488b-87b4-717c0dbcf027 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:03.190188 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f62f3a-b29e-4533-b558-3c03b371756d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:03.190814 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f62f3a-b29e-4533-b558-3c03b371756d None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:03.191092 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f62f3a-b29e-4533-b558-3c03b371756d None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:03.191361 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f62f3a-b29e-4533-b558-3c03b371756d None admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:03.192084 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f62f3a-b29e-4533-b558-3c03b371756d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:03.192389 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f62f3a-b29e-4533-b558-3c03b371756d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:03.194282 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f62f3a-b29e-4533-b558-3c03b371756d None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:03.206950 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 429/870] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:39:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:03.213465 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daefa1e-acac-40a5-a0fa-807152ba87d1 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:03.310056 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:03.324034 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daefa1e-acac-40a5-a0fa-807152ba87d1 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:03.360619 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27fbd01-4a09-4770-bd54-953bd98a18a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:03.361232 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27fbd01-4a09-4770-bd54-953bd98a18a6 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:03.361517 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27fbd01-4a09-4770-bd54-953bd98a18a6 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:03.361849 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27fbd01-4a09-4770-bd54-953bd98a18a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:03.369668 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27fbd01-4a09-4770-bd54-953bd98a18a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:03.371129 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27fbd01-4a09-4770-bd54-953bd98a18a6 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:03.384032 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-b27fbd01-4a09-4770-bd54-953bd98a18a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:39:03.384032 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:03.384032 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:03.384032 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:03.384032 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:03.384032 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:03.384032 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:03.384032 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:03.384032 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:03.384032 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:03.384032 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:03.384032 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:03.384032 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 12:39:03.384032 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 12:39:03.384032 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 12:39:03.384964 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 12:39:03.384964 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:03.384964 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:03.384964 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:03.384964 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:03.384964 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:03.384964 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:03.384964 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:03.384964 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:03.384964 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:03.384964 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:03.384964 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:03.384964 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:03.384964 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:03.384964 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:03.384964 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:03.384964 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:03.385840 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 12:39:03.385840 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 12:39:03.385840 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 12:39:03.385840 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 12:39:03.385840 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 12:39:03.385840 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 12:39:03.385840 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:39:03.385840 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:03.390031 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 442/871] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:39:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:39:03.391401 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f62f3a-b29e-4533-b558-3c03b371756d None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:03.500761 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f62f3a-b29e-4533-b558-3c03b371756d None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:03.514368 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:03.536862 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfd858a-9e23-4646-9c11-a191354299d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:03.537845 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfd858a-9e23-4646-9c11-a191354299d0 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:03.538259 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfd858a-9e23-4646-9c11-a191354299d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:03.538651 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfd858a-9e23-4646-9c11-a191354299d0 None admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:03.539679 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfd858a-9e23-4646-9c11-a191354299d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:03.540026 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfd858a-9e23-4646-9c11-a191354299d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:03.541504 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfd858a-9e23-4646-9c11-a191354299d0 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:03.554920 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 430/872] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 26 12:39:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:03.560679 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27fbd01-4a09-4770-bd54-953bd98a18a6 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:03.674821 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27fbd01-4a09-4770-bd54-953bd98a18a6 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:03.711300 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6847c0-a3e6-4558-959b-ddd73f2a38ee None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:03.711615 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6847c0-a3e6-4558-959b-ddd73f2a38ee None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:03.711872 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6847c0-a3e6-4558-959b-ddd73f2a38ee None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:03.713416 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 431/873] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:39:03.717413 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faa5593-0efb-4de3-ad6e-3840ef10eab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:03.717413 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa5593-0efb-4de3-ad6e-3840ef10eab6 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:03.717413 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa5593-0efb-4de3-ad6e-3840ef10eab6 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:03.717413 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa5593-0efb-4de3-ad6e-3840ef10eab6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:03.717757 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faa5593-0efb-4de3-ad6e-3840ef10eab6 None admin] RBAC: Adding query filter params (role.id=9c68629183974be9bfb1ed95f69912ee, scope.project.id=644278e7e8934b37a1e7ab20d35764cd, user.id=30fc532f464843a3ad35e4d3c387bb3a) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:03.718034 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faa5593-0efb-4de3-ad6e-3840ef10eab6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:03.719214 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa9e94-34aa-4be9-9f0d-69dc15c097d5 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:03.719319 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faa5593-0efb-4de3-ad6e-3840ef10eab6 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:03.719479 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa9e94-34aa-4be9-9f0d-69dc15c097d5 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:03.719782 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa9e94-34aa-4be9-9f0d-69dc15c097d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:03.726615 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 443/874] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:39:03 2022] GET /identity/v3/role_assignments?user.id=30fc532f464843a3ad35e4d3c387bb3a&scope.project.id=644278e7e8934b37a1e7ab20d35764cd&role.id=9c68629183974be9bfb1ed95f69912ee => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:03.737950 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722ea86-f9d3-4846-848f-a449ced9ac18 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:03.738540 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722ea86-f9d3-4846-848f-a449ced9ac18 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:03.739034 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722ea86-f9d3-4846-848f-a449ced9ac18 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:03.741424 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 444/875] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 12:39:03.744831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-dbfa9e94-34aa-4be9-9f0d-69dc15c097d5 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:03.746559 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-dbfa9e94-34aa-4be9-9f0d-69dc15c097d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:03.746725 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35bbc0-7e0c-4d1e-a3e0-445df910e2ca None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:03.747073 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35bbc0-7e0c-4d1e-a3e0-445df910e2ca None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:03.747449 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35bbc0-7e0c-4d1e-a3e0-445df910e2ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:03.768658 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-9b35bbc0-7e0c-4d1e-a3e0-445df910e2ca None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:03.770003 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-9b35bbc0-7e0c-4d1e-a3e0-445df910e2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:03.779527 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:03.849952 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 432/876] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:03 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:39:03.854093 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 445/877] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:03 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 12:39:03.873465 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfa9e94-34aa-4be9-9f0d-69dc15c097d5 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:03.876942 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b35bbc0-7e0c-4d1e-a3e0-445df910e2ca None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:03.999910 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfa9e94-34aa-4be9-9f0d-69dc15c097d5 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:04.001861 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:04.017987 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b35bbc0-7e0c-4d1e-a3e0-445df910e2ca None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:04.044202 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88deb99e-6a56-46de-afe6-ef43ee86b8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:04.045137 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88deb99e-6a56-46de-afe6-ef43ee86b8cb None admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:04.046123 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88deb99e-6a56-46de-afe6-ef43ee86b8cb None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:04.046586 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88deb99e-6a56-46de-afe6-ef43ee86b8cb None admin] PATH_INFO: `/v3/services` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:04.047816 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88deb99e-6a56-46de-afe6-ef43ee86b8cb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:04.050325 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88deb99e-6a56-46de-afe6-ef43ee86b8cb None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:04.061578 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e87b7a-b0bd-4fc5-b0fb-92d60ec83d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:04.062295 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e87b7a-b0bd-4fc5-b0fb-92d60ec83d00 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:04.062588 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e87b7a-b0bd-4fc5-b0fb-92d60ec83d00 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:04.062855 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e87b7a-b0bd-4fc5-b0fb-92d60ec83d00 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:04.063509 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e87b7a-b0bd-4fc5-b0fb-92d60ec83d00 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:04.065106 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e87b7a-b0bd-4fc5-b0fb-92d60ec83d00 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:04.072575 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 433/878] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Jul 26 12:39:03 2022] POST /identity/v3/services => generated 269 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:39:04.074678 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-b1e87b7a-b0bd-4fc5-b0fb-92d60ec83d00 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 12:39:04.074678 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:04.074678 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 12:39:04.074678 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 12:39:04.074678 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 12:39:04.074678 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 12:39:04.074678 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 12:39:04.074678 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:39:04.074678 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 12:39:04.074678 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:04.074678 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 12:39:04.074678 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:04.074678 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:04.074678 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:04.074678 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:04.074678 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:04.074678 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:04.075721 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:04.075721 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:04.075721 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:04.075721 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:04.075721 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:04.075721 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:04.075721 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 12:39:04.075721 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 12:39:04.075721 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 12:39:04.075721 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 12:39:04.075721 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:04.075721 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:04.075721 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:04.075721 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:04.075721 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:04.075721 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:04.076958 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:04.076958 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:04.076958 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:04.076958 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:04.076958 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:04.076958 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:04.076958 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:04.076958 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:04.076958 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:04.076958 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:04.076958 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 12:39:04.076958 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:39:04.076958 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 12:39:04.076958 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:04.076958 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 446/879] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Jul 26 12:39:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:39:04.082609 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88deb99e-6a56-46de-afe6-ef43ee86b8cb None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:04.206956 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88deb99e-6a56-46de-afe6-ef43ee86b8cb None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:04.244866 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f76a0e-ac90-4f4e-adb3-62e0ec21f407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:04.245502 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f76a0e-ac90-4f4e-adb3-62e0ec21f407 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:04.245786 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f76a0e-ac90-4f4e-adb3-62e0ec21f407 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:04.246062 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f76a0e-ac90-4f4e-adb3-62e0ec21f407 None admin] PATH_INFO: `/v3/services` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:04.246754 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f76a0e-ac90-4f4e-adb3-62e0ec21f407 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:04.247062 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f76a0e-ac90-4f4e-adb3-62e0ec21f407 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:04.248476 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f76a0e-ac90-4f4e-adb3-62e0ec21f407 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:04.254012 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 434/880] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Jul 26 12:39:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:04.260527 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e87b7a-b0bd-4fc5-b0fb-92d60ec83d00 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:04.270104 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:04.371442 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e87b7a-b0bd-4fc5-b0fb-92d60ec83d00 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:04.407573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76331475-79e5-48fa-a413-b2fe149ab2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:04.408531 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76331475-79e5-48fa-a413-b2fe149ab2fc None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:04.408988 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76331475-79e5-48fa-a413-b2fe149ab2fc None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:04.409470 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76331475-79e5-48fa-a413-b2fe149ab2fc None admin] PATH_INFO: `/v3/services` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:04.410326 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76331475-79e5-48fa-a413-b2fe149ab2fc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:04.410802 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76331475-79e5-48fa-a413-b2fe149ab2fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:04.412409 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76331475-79e5-48fa-a413-b2fe149ab2fc None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:04.417632 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 447/881] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Jul 26 12:39:04 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:04.848254 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba7144-64df-4a7e-a2ff-6257f524b886 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:04.848638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba7144-64df-4a7e-a2ff-6257f524b886 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:04.848859 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba7144-64df-4a7e-a2ff-6257f524b886 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:04.850549 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 435/882] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:39:04.855926 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6432d921-f058-494c-86bf-372283a269b3 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:04.856219 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6432d921-f058-494c-86bf-372283a269b3 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:04.856496 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6432d921-f058-494c-86bf-372283a269b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:04.925031 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-6432d921-f058-494c-86bf-372283a269b3 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:04.926245 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-6432d921-f058-494c-86bf-372283a269b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:04.992006 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:05.003989 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 448/883] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:04 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:39:05.018777 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba7144-64df-4a7e-a2ff-6257f524b886 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:05.044572 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:05.127851 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba7144-64df-4a7e-a2ff-6257f524b886 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:05.162697 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aaca8c-654d-40bf-bd36-dbbbe635257d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:05.163349 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aaca8c-654d-40bf-bd36-dbbbe635257d None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:05.163624 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aaca8c-654d-40bf-bd36-dbbbe635257d None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:05.163920 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aaca8c-654d-40bf-bd36-dbbbe635257d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:05.164551 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aaca8c-654d-40bf-bd36-dbbbe635257d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:05.165923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aaca8c-654d-40bf-bd36-dbbbe635257d None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:05.175330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-01aaca8c-654d-40bf-bd36-dbbbe635257d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 12:39:05.175330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:05.175330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 12:39:05.175330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 12:39:05.175330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 12:39:05.175330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 12:39:05.175330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 12:39:05.175330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:39:05.175330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 12:39:05.175330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:05.175330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 12:39:05.175330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:05.175330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:05.175330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:05.175330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:05.175330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:05.175330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:05.176369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:05.176369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:05.176369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:05.176369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:05.176369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:05.176369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:05.176369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 12:39:05.176369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 12:39:05.176369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 12:39:05.176369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 12:39:05.176369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:05.176369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:05.176369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:05.176369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:05.176369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:05.176369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:05.177294 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:05.177294 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:05.177294 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:05.177294 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:05.177294 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:05.177294 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:05.177294 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:05.177294 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:05.177294 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:05.177294 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:05.177294 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 12:39:05.177294 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:39:05.177294 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 12:39:05.177294 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:05.181487 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 436/884] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 26 12:39:05 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:39:05.182898 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6432d921-f058-494c-86bf-372283a269b3 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:05.288938 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6432d921-f058-494c-86bf-372283a269b3 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:05.326183 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca5ee7c-8f6a-4da3-b48e-71468ecd6944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:05.326810 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5ee7c-8f6a-4da3-b48e-71468ecd6944 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:05.327091 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5ee7c-8f6a-4da3-b48e-71468ecd6944 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:05.327371 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5ee7c-8f6a-4da3-b48e-71468ecd6944 None admin] PATH_INFO: `/v3/services` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:05.328232 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca5ee7c-8f6a-4da3-b48e-71468ecd6944 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:05.328542 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca5ee7c-8f6a-4da3-b48e-71468ecd6944 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:05.329931 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca5ee7c-8f6a-4da3-b48e-71468ecd6944 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:05.335740 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 449/885] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 26 12:39:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:05.340971 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aaca8c-654d-40bf-bd36-dbbbe635257d None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:05.427704 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:05.444285 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aaca8c-654d-40bf-bd36-dbbbe635257d None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:05.482970 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a591f6f5-371b-49f7-b1fd-f95a6b6a01ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:05.483577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a591f6f5-371b-49f7-b1fd-f95a6b6a01ac None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:05.483903 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a591f6f5-371b-49f7-b1fd-f95a6b6a01ac None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:05.484176 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a591f6f5-371b-49f7-b1fd-f95a6b6a01ac None admin] PATH_INFO: `/v3/services` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:05.484845 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a591f6f5-371b-49f7-b1fd-f95a6b6a01ac None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:05.485148 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a591f6f5-371b-49f7-b1fd-f95a6b6a01ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:05.486510 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a591f6f5-371b-49f7-b1fd-f95a6b6a01ac None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:05.492430 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 437/886] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 26 12:39:05 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:05.498207 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca5ee7c-8f6a-4da3-b48e-71468ecd6944 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:05.574266 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:05.615689 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca5ee7c-8f6a-4da3-b48e-71468ecd6944 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:05.654447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc0d2b0-74e6-42ca-b98c-1b6b4c946ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:05.655069 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc0d2b0-74e6-42ca-b98c-1b6b4c946ff4 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:05.655348 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc0d2b0-74e6-42ca-b98c-1b6b4c946ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:05.655622 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc0d2b0-74e6-42ca-b98c-1b6b4c946ff4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:05.656412 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc0d2b0-74e6-42ca-b98c-1b6b4c946ff4 None admin] RBAC: Adding query filter params (interface=public, service_id=887a13f8ab104ca7b06790a4f29ac457, region_id=RegionOne) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:05.656721 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc0d2b0-74e6-42ca-b98c-1b6b4c946ff4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:05.658096 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc0d2b0-74e6-42ca-b98c-1b6b4c946ff4 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:05.666534 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 450/887] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Jul 26 12:39:05 2022] GET /identity/v3/endpoints?service_id=887a13f8ab104ca7b06790a4f29ac457&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:05.671709 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a591f6f5-371b-49f7-b1fd-f95a6b6a01ac None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:05.772410 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a591f6f5-371b-49f7-b1fd-f95a6b6a01ac None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:05.811507 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3becf98a-fdf0-4b95-b803-ba674790cb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:05.812163 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3becf98a-fdf0-4b95-b803-ba674790cb41 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:05.812441 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3becf98a-fdf0-4b95-b803-ba674790cb41 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:05.812708 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3becf98a-fdf0-4b95-b803-ba674790cb41 None admin] PATH_INFO: `/v3/services` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:05.813354 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3becf98a-fdf0-4b95-b803-ba674790cb41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:05.814712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3becf98a-fdf0-4b95-b803-ba674790cb41 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:05.821615 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 438/888] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Jul 26 12:39:05 2022] GET /identity/v3/services => generated 1268 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:39:06.061535 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961bbb2f-2cdc-411d-8441-e1093a5947ba None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:06.061950 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961bbb2f-2cdc-411d-8441-e1093a5947ba None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:06.062330 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961bbb2f-2cdc-411d-8441-e1093a5947ba None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:06.064132 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 451/889] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:39:06.070098 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40902a-04c1-4d3b-98a7-a6482908ab2a None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:06.070497 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40902a-04c1-4d3b-98a7-a6482908ab2a None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:06.070861 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40902a-04c1-4d3b-98a7-a6482908ab2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:06.089433 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-1f40902a-04c1-4d3b-98a7-a6482908ab2a None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:06.090630 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-1f40902a-04c1-4d3b-98a7-a6482908ab2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:06.101177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:06.167763 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 439/890] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:06 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:39:06.185481 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961bbb2f-2cdc-411d-8441-e1093a5947ba None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:06.269354 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef6f65-5ac0-4a8e-a6b7-46b7bc0b7d68 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:06.269698 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef6f65-5ac0-4a8e-a6b7-46b7bc0b7d68 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:06.269911 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef6f65-5ac0-4a8e-a6b7-46b7bc0b7d68 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:06.272078 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 440/891] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 12:39:06.277783 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1410bd-77e3-434b-af99-d097c280ecb2 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:06.278321 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1410bd-77e3-434b-af99-d097c280ecb2 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:06.278797 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1410bd-77e3-434b-af99-d097c280ecb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:06.297207 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-2c1410bd-77e3-434b-af99-d097c280ecb2 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:06.298748 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-2c1410bd-77e3-434b-af99-d097c280ecb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:06.304219 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961bbb2f-2cdc-411d-8441-e1093a5947ba None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:06.348203 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:06.359623 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa844857-afc1-448e-840f-85e987d05aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:06.360789 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa844857-afc1-448e-840f-85e987d05aca None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:06.361234 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa844857-afc1-448e-840f-85e987d05aca None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:06.361655 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa844857-afc1-448e-840f-85e987d05aca None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:06.376930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa844857-afc1-448e-840f-85e987d05aca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:39:06.380078 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 441/892] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:06 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:39:06.383141 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-aa844857-afc1-448e-840f-85e987d05aca None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 12:39:06.383141 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:06.383141 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:06.383141 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:06.383141 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:06.383141 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:06.383141 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:06.383141 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:06.383141 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:06.383141 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:06.383141 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:06.383141 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:06.383141 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:39:06.383141 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:39:06.383141 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:39:06.383141 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:39:06.384268 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:39:06.384268 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:39:06.384268 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:39:06.384268 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:39:06.384268 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:06.384268 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:06.384268 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:06.384268 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:06.384268 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:06.384268 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:06.384268 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:06.384268 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:06.384268 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:06.384268 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:06.384268 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:06.384268 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:06.385287 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:06.385287 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:06.385287 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:06.385287 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:06.385287 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:39:06.385287 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:39:06.385287 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:39:06.385287 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:39:06.385287 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:39:06.385287 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:39:06.385287 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 12:39:06.385287 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:06.392143 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 452/893] 10.222.0.20 () {56 vars in 1120 bytes} [Tue Jul 26 12:39:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:39:06.393684 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1410bd-77e3-434b-af99-d097c280ecb2 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:06.402228 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa844857-afc1-448e-840f-85e987d05aca None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:06.478603 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:06.506891 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1410bd-77e3-434b-af99-d097c280ecb2 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:06.514136 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa844857-afc1-448e-840f-85e987d05aca None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:06.546186 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b92db92-34b2-4d41-88b2-acbc136e8de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:06.547188 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b92db92-34b2-4d41-88b2-acbc136e8de5 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:06.547763 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b92db92-34b2-4d41-88b2-acbc136e8de5 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:06.548181 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b92db92-34b2-4d41-88b2-acbc136e8de5 None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:06.549107 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b92db92-34b2-4d41-88b2-acbc136e8de5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:06.549699 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b92db92-34b2-4d41-88b2-acbc136e8de5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:06.551244 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b92db92-34b2-4d41-88b2-acbc136e8de5 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:06.553881 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e2ba79-dd12-4cad-a58a-bc1fe62b084e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:06.554694 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2ba79-dd12-4cad-a58a-bc1fe62b084e None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:06.555280 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2ba79-dd12-4cad-a58a-bc1fe62b084e None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:06.555533 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2ba79-dd12-4cad-a58a-bc1fe62b084e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:06.556361 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e2ba79-dd12-4cad-a58a-bc1fe62b084e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:06.557924 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e2ba79-dd12-4cad-a58a-bc1fe62b084e None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:06.567709 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 442/894] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Jul 26 12:39:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:06.568342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-55e2ba79-dd12-4cad-a58a-bc1fe62b084e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 12:39:06.568342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:06.568342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 12:39:06.568342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 12:39:06.568342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 12:39:06.568342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 12:39:06.568342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 12:39:06.568342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:39:06.568342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 12:39:06.568342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:06.568342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 12:39:06.568342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:06.568342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:06.568342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:06.568342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:06.568342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:06.569459 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:06.569459 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:06.569459 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:06.569459 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:06.569459 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:06.569459 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:06.569459 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:06.569459 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 12:39:06.569459 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 12:39:06.569459 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 12:39:06.569459 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 12:39:06.569459 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:06.569459 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:06.569459 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:06.569459 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:06.569459 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:06.570501 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:06.570501 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:06.570501 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:06.570501 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:06.570501 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:06.570501 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:06.570501 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:06.570501 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:06.570501 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:06.570501 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:06.570501 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:06.570501 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 12:39:06.570501 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:39:06.570501 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 12:39:06.570501 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:06.572502 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b92db92-34b2-4d41-88b2-acbc136e8de5 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:06.575705 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 453/895] 10.222.0.20 () {56 vars in 1132 bytes} [Tue Jul 26 12:39:06 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:39:06.578878 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e2ba79-dd12-4cad-a58a-bc1fe62b084e None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:06.650007 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:06.689405 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b92db92-34b2-4d41-88b2-acbc136e8de5 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:06.700395 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e2ba79-dd12-4cad-a58a-bc1fe62b084e None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:06.727586 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc169a59-9b3a-4ae0-bf4e-bf2467eae1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:06.728285 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc169a59-9b3a-4ae0-bf4e-bf2467eae1e6 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:06.728573 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc169a59-9b3a-4ae0-bf4e-bf2467eae1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:06.728862 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc169a59-9b3a-4ae0-bf4e-bf2467eae1e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:06.732038 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:06.741590 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc169a59-9b3a-4ae0-bf4e-bf2467eae1e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:06.743055 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc169a59-9b3a-4ae0-bf4e-bf2467eae1e6 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:06.743969 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d7b05-d5e5-4a5a-aabb-bd143793197c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:06.744583 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d7b05-d5e5-4a5a-aabb-bd143793197c None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:06.744877 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d7b05-d5e5-4a5a-aabb-bd143793197c None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:06.745133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d7b05-d5e5-4a5a-aabb-bd143793197c None admin] PATH_INFO: `/v3/services` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:06.745818 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695d7b05-d5e5-4a5a-aabb-bd143793197c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:06.746123 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695d7b05-d5e5-4a5a-aabb-bd143793197c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:06.747470 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695d7b05-d5e5-4a5a-aabb-bd143793197c None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:06.752688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 454/896] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Jul 26 12:39:06 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:06.757437 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84f244-f0e9-48c8-bc29-368495fa0f9f None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:06.757739 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84f244-f0e9-48c8-bc29-368495fa0f9f None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:06.758021 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84f244-f0e9-48c8-bc29-368495fa0f9f None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:06.758540 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-dc169a59-9b3a-4ae0-bf4e-bf2467eae1e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:39:06.758540 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:06.758540 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:06.758540 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:06.758540 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:06.758540 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:06.758540 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:06.758540 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:06.758540 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:06.758540 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:06.758540 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:06.758540 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:06.758540 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:39:06.758540 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:39:06.758540 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:39:06.759380 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:39:06.759380 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:06.759380 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:06.759380 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:06.759380 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:06.759380 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:06.759380 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:06.759380 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:06.759380 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:06.759380 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:06.759380 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:06.759380 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:06.759380 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:06.759380 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:06.759380 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:06.759380 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:06.759380 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:06.760319 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:39:06.760319 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:39:06.760319 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:39:06.760319 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:06.763555 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 455/897] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:06 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:39:06.764395 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d84f244-f0e9-48c8-bc29-368495fa0f9f None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:06.764595 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 443/898] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:39:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:39:06.765828 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128db93-68b6-4a12-aa90-0acd1159e1a8 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:06.766118 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128db93-68b6-4a12-aa90-0acd1159e1a8 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:06.766396 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128db93-68b6-4a12-aa90-0acd1159e1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:06.787405 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-8128db93-68b6-4a12-aa90-0acd1159e1a8 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:06.789931 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-8128db93-68b6-4a12-aa90-0acd1159e1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:06.887776 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 444/899] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:06 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:39:06.888677 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8128db93-68b6-4a12-aa90-0acd1159e1a8 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:06.889614 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d84f244-f0e9-48c8-bc29-368495fa0f9f None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:06.908645 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:06.928151 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0f8a9-d620-4580-b541-851b8b047014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:06.928717 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0f8a9-d620-4580-b541-851b8b047014 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:06.929000 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0f8a9-d620-4580-b541-851b8b047014 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:06.929273 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0f8a9-d620-4580-b541-851b8b047014 None admin] PATH_INFO: `/v3/services` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:06.929947 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee0f8a9-d620-4580-b541-851b8b047014 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:06.930253 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee0f8a9-d620-4580-b541-851b8b047014 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:06.931615 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee0f8a9-d620-4580-b541-851b8b047014 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:06.937038 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 456/900] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Jul 26 12:39:06 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:06.937652 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0f8a9-d620-4580-b541-851b8b047014 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:07.010710 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8128db93-68b6-4a12-aa90-0acd1159e1a8 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:07.030017 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:07.051581 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a2778b-364a-45c8-b208-adfa65902936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:07.053075 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a2778b-364a-45c8-b208-adfa65902936 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:07.053555 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a2778b-364a-45c8-b208-adfa65902936 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:07.053721 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0f8a9-d620-4580-b541-851b8b047014 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:07.054195 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a2778b-364a-45c8-b208-adfa65902936 None admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:07.055091 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a2778b-364a-45c8-b208-adfa65902936 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:07.055399 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a2778b-364a-45c8-b208-adfa65902936 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:07.056823 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a2778b-364a-45c8-b208-adfa65902936 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:07.068740 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 445/901] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:39:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:07.069550 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a2778b-364a-45c8-b208-adfa65902936 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:07.096839 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c80bf5-1ab4-4c3b-9373-fad50af7efd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:07.097458 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c80bf5-1ab4-4c3b-9373-fad50af7efd5 None admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:07.097747 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c80bf5-1ab4-4c3b-9373-fad50af7efd5 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:07.098024 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c80bf5-1ab4-4c3b-9373-fad50af7efd5 None admin] PATH_INFO: `/v3/services` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:07.098720 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c80bf5-1ab4-4c3b-9373-fad50af7efd5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:07.100698 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c80bf5-1ab4-4c3b-9373-fad50af7efd5 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:07.127430 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 457/902] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Jul 26 12:39:06 2022] POST /identity/v3/services => generated 241 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:39:07.127430 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c80bf5-1ab4-4c3b-9373-fad50af7efd5 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:07.189730 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:07.195569 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a2778b-364a-45c8-b208-adfa65902936 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:07.244372 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951859e5-a6b8-4d92-b1d0-9c0d89946b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:07.245078 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951859e5-a6b8-4d92-b1d0-9c0d89946b97 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:07.245356 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951859e5-a6b8-4d92-b1d0-9c0d89946b97 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:07.245625 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951859e5-a6b8-4d92-b1d0-9c0d89946b97 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:07.246385 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951859e5-a6b8-4d92-b1d0-9c0d89946b97 None admin] RBAC: Adding query filter params (interface=public, service_id=511f6b27d24345a4ae62e4be7417ef81, region_id=RegionOne) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:07.246694 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951859e5-a6b8-4d92-b1d0-9c0d89946b97 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:07.248380 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951859e5-a6b8-4d92-b1d0-9c0d89946b97 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:07.254663 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 446/903] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Jul 26 12:39:07 2022] GET /identity/v3/endpoints?service_id=511f6b27d24345a4ae62e4be7417ef81&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:07.260969 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951859e5-a6b8-4d92-b1d0-9c0d89946b97 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:07.263849 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c80bf5-1ab4-4c3b-9373-fad50af7efd5 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:07.303010 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c2e36-c7aa-45bc-9e1d-29f6272d6c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:07.303676 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c2e36-c7aa-45bc-9e1d-29f6272d6c6b None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:07.303963 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c2e36-c7aa-45bc-9e1d-29f6272d6c6b None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:07.304238 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c2e36-c7aa-45bc-9e1d-29f6272d6c6b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:07.307610 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:07.313561 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3c2e36-c7aa-45bc-9e1d-29f6272d6c6b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:07.315930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3c2e36-c7aa-45bc-9e1d-29f6272d6c6b None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:07.331470 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-8c3c2e36-c7aa-45bc-9e1d-29f6272d6c6b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 12:39:07.331470 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:07.331470 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:07.331470 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:07.331470 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:07.331470 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:07.331470 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:07.331470 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:07.331470 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:07.331470 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:07.331470 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:07.331470 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:07.331470 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 12:39:07.331470 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 12:39:07.331470 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 12:39:07.331470 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 12:39:07.332777 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:07.332777 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:07.332777 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 12:39:07.332777 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:39:07.332777 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 12:39:07.332777 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:39:07.332777 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:07.332777 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:07.332777 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:07.332777 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:07.332777 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:07.332777 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:07.332777 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:07.332777 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:07.332777 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:07.332777 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:07.332777 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:07.332777 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:07.334002 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:07.334002 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:07.334002 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 12:39:07.334002 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 12:39:07.334002 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 12:39:07.334002 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 12:39:07.334002 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 12:39:07.334002 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 12:39:07.334002 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 12:39:07.334002 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:07.334585 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 458/904] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Jul 26 12:39:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:39:07.340681 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c2e36-c7aa-45bc-9e1d-29f6272d6c6b None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:07.388012 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951859e5-a6b8-4d92-b1d0-9c0d89946b97 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:07.429966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121df7f1-5403-4913-b072-c800a5f7ba83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:07.430650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121df7f1-5403-4913-b072-c800a5f7ba83 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:07.430937 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121df7f1-5403-4913-b072-c800a5f7ba83 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:07.431270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121df7f1-5403-4913-b072-c800a5f7ba83 None admin] PATH_INFO: `/v3/services` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:07.432046 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121df7f1-5403-4913-b072-c800a5f7ba83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:07.433575 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121df7f1-5403-4913-b072-c800a5f7ba83 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:07.440948 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 447/905] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Jul 26 12:39:07 2022] GET /identity/v3/services => generated 1497 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 12:39:07.471390 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c2e36-c7aa-45bc-9e1d-29f6272d6c6b None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:07.475037 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:07.511622 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68d29ff-9f3f-4628-9088-2b23cc7351cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:07.512292 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d29ff-9f3f-4628-9088-2b23cc7351cf None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:07.512571 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d29ff-9f3f-4628-9088-2b23cc7351cf None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:07.512851 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d29ff-9f3f-4628-9088-2b23cc7351cf None admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:07.513677 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68d29ff-9f3f-4628-9088-2b23cc7351cf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:07.513987 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68d29ff-9f3f-4628-9088-2b23cc7351cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:07.515467 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68d29ff-9f3f-4628-9088-2b23cc7351cf None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:07.537042 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 459/906] 10.222.0.20 () {56 vars in 1149 bytes} [Tue Jul 26 12:39:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:39:07.542761 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121df7f1-5403-4913-b072-c800a5f7ba83 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:07.653273 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121df7f1-5403-4913-b072-c800a5f7ba83 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:07.655045 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:07.692629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69414620-87ca-48f6-ad3c-c871922f50f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:07.693230 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69414620-87ca-48f6-ad3c-c871922f50f9 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:07.693517 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69414620-87ca-48f6-ad3c-c871922f50f9 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:07.693792 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69414620-87ca-48f6-ad3c-c871922f50f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:07.766794 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69414620-87ca-48f6-ad3c-c871922f50f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:07.768280 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69414620-87ca-48f6-ad3c-c871922f50f9 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:07.877255 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 448/907] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:39:07 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 12:39:07.882731 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68d29ff-9f3f-4628-9088-2b23cc7351cf None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:07.891323 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5ea7e-03eb-459c-a17a-2fc85cac63c5 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:07.891928 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5ea7e-03eb-459c-a17a-2fc85cac63c5 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:07.892420 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5ea7e-03eb-459c-a17a-2fc85cac63c5 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:07.894259 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 449/908] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 12:39:07.900515 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8464422-963e-4f2a-a987-4ed0e2192250 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:07.901171 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8464422-963e-4f2a-a987-4ed0e2192250 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:07.901656 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8464422-963e-4f2a-a987-4ed0e2192250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:07.908802 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:07.923319 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-f8464422-963e-4f2a-a987-4ed0e2192250 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:07.924960 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-f8464422-963e-4f2a-a987-4ed0e2192250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:07.994572 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68d29ff-9f3f-4628-9088-2b23cc7351cf None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:08.003304 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 450/909] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 12:39:08.025332 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8464422-963e-4f2a-a987-4ed0e2192250 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:08.034803 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537cce5-56fc-428d-8f25-c8a01b3ddd95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:08.035491 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537cce5-56fc-428d-8f25-c8a01b3ddd95 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:08.035803 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537cce5-56fc-428d-8f25-c8a01b3ddd95 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:08.036074 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537cce5-56fc-428d-8f25-c8a01b3ddd95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:08.048328 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1537cce5-56fc-428d-8f25-c8a01b3ddd95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:08.049790 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1537cce5-56fc-428d-8f25-c8a01b3ddd95 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:08.065148 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-1537cce5-56fc-428d-8f25-c8a01b3ddd95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:39:08.065148 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:08.065148 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:08.065148 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:08.065148 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:08.065148 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:08.065148 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:08.065148 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:08.065148 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:08.065148 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:08.065148 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:08.065148 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:08.065148 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:39:08.065148 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:39:08.065148 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:39:08.066064 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:39:08.066064 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:08.066064 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:08.066064 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:08.066064 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:08.066064 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:08.066064 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:08.066064 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:08.066064 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:08.066064 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:08.066064 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:08.066064 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:08.066064 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:08.066064 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:08.066064 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:08.066064 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:08.066064 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:08.067538 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:39:08.067538 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:39:08.067538 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:39:08.067538 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:08.067538 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 460/910] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:39:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:39:08.072407 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537cce5-56fc-428d-8f25-c8a01b3ddd95 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:08.092029 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:08.150143 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8464422-963e-4f2a-a987-4ed0e2192250 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:08.189128 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:08.190412 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537cce5-56fc-428d-8f25-c8a01b3ddd95 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:08.197219 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43aaa0b-f39b-4bda-8909-c733fb81162c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:08.197869 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43aaa0b-f39b-4bda-8909-c733fb81162c None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:08.198153 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43aaa0b-f39b-4bda-8909-c733fb81162c None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:08.198427 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43aaa0b-f39b-4bda-8909-c733fb81162c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:08.199067 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43aaa0b-f39b-4bda-8909-c733fb81162c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:08.200471 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43aaa0b-f39b-4bda-8909-c733fb81162c None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:08.213232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-c43aaa0b-f39b-4bda-8909-c733fb81162c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 12:39:08.213232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:08.213232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 12:39:08.213232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 12:39:08.213232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 12:39:08.213232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 12:39:08.213232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 12:39:08.213232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:39:08.213232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 12:39:08.213232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:08.213232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 12:39:08.213232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:08.213232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:08.213232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:08.213232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:08.213232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:08.213232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:08.214309 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:08.214309 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:08.214309 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:08.214309 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:08.214309 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:08.214309 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:08.214309 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 12:39:08.214309 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 12:39:08.214309 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 12:39:08.214309 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 12:39:08.214309 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:08.214309 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:08.214309 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:08.214309 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:08.214309 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:08.214309 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:08.215142 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:08.215142 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:08.215142 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:08.215142 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:08.215142 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:08.215142 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:08.215142 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:08.215142 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:08.215142 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:08.215142 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:08.215142 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 12:39:08.215142 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:39:08.215142 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 12:39:08.215142 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:08.220033 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 451/911] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 26 12:39:08 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:39:08.221671 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43aaa0b-f39b-4bda-8909-c733fb81162c None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:08.241712 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ced15-0e46-4693-9b80-3cc79290c804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:08.242896 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ced15-0e46-4693-9b80-3cc79290c804 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:08.243190 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ced15-0e46-4693-9b80-3cc79290c804 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:08.243470 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ced15-0e46-4693-9b80-3cc79290c804 None admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:08.244381 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201ced15-0e46-4693-9b80-3cc79290c804 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:08.244879 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201ced15-0e46-4693-9b80-3cc79290c804 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:08.246511 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201ced15-0e46-4693-9b80-3cc79290c804 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:08.266342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 461/912] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:39:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:08.271471 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ced15-0e46-4693-9b80-3cc79290c804 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:08.355125 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:08.360366 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43aaa0b-f39b-4bda-8909-c733fb81162c None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:08.390975 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ced15-0e46-4693-9b80-3cc79290c804 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:08.398311 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20ec329-3aff-456d-a73c-cde8ddaddf49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:08.399539 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ec329-3aff-456d-a73c-cde8ddaddf49 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:08.400059 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ec329-3aff-456d-a73c-cde8ddaddf49 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:08.400529 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ec329-3aff-456d-a73c-cde8ddaddf49 None admin] PATH_INFO: `/v3/services` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:08.401402 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20ec329-3aff-456d-a73c-cde8ddaddf49 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:08.401922 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20ec329-3aff-456d-a73c-cde8ddaddf49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:08.403485 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20ec329-3aff-456d-a73c-cde8ddaddf49 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:08.409073 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 452/913] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 26 12:39:08 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:39:08.414438 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20ec329-3aff-456d-a73c-cde8ddaddf49 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:08.433325 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996354ab-26b2-4971-aa0d-235bc87239a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:08.434006 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996354ab-26b2-4971-aa0d-235bc87239a7 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:08.434288 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996354ab-26b2-4971-aa0d-235bc87239a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:08.434560 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996354ab-26b2-4971-aa0d-235bc87239a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:08.441967 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996354ab-26b2-4971-aa0d-235bc87239a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:08.443573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996354ab-26b2-4971-aa0d-235bc87239a7 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:08.455490 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-996354ab-26b2-4971-aa0d-235bc87239a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:39:08.455490 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:08.455490 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:08.455490 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:08.455490 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:08.455490 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:08.455490 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:08.455490 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:08.455490 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:08.455490 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:08.455490 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:08.455490 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:08.455490 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 12:39:08.455490 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 12:39:08.455490 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 12:39:08.456411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 12:39:08.456411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:08.456411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:08.456411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:08.456411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:08.456411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:08.456411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:08.456411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:08.456411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:08.456411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:08.456411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:08.456411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:08.456411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:08.456411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:08.456411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:08.456411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:08.456411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:08.457622 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 12:39:08.457622 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 12:39:08.457622 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 12:39:08.457622 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 12:39:08.457622 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 12:39:08.457622 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 12:39:08.457622 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:39:08.457622 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:08.457622 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 462/914] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:39:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:39:08.463008 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996354ab-26b2-4971-aa0d-235bc87239a7 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:08.484605 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:08.532343 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20ec329-3aff-456d-a73c-cde8ddaddf49 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:08.570300 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d04f3-5b28-4744-adcc-e09f83ee203e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:08.571384 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d04f3-5b28-4744-adcc-e09f83ee203e None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:08.571946 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d04f3-5b28-4744-adcc-e09f83ee203e None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:08.572500 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d04f3-5b28-4744-adcc-e09f83ee203e None admin] PATH_INFO: `/v3/services` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:08.573424 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092d04f3-5b28-4744-adcc-e09f83ee203e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:08.573926 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092d04f3-5b28-4744-adcc-e09f83ee203e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:08.575737 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092d04f3-5b28-4744-adcc-e09f83ee203e None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:08.581745 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 453/915] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 26 12:39:08 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:39:08.590762 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d04f3-5b28-4744-adcc-e09f83ee203e None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:08.593331 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996354ab-26b2-4971-aa0d-235bc87239a7 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:08.635530 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:08.645039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b268da-c42f-4909-883f-7a975ee12abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:08.646561 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b268da-c42f-4909-883f-7a975ee12abb None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:08.646994 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b268da-c42f-4909-883f-7a975ee12abb None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:08.648150 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b268da-c42f-4909-883f-7a975ee12abb None admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:08.649401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b268da-c42f-4909-883f-7a975ee12abb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:08.649895 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b268da-c42f-4909-883f-7a975ee12abb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:08.652050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b268da-c42f-4909-883f-7a975ee12abb None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:08.667694 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 463/916] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 26 12:39:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:08.673277 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b268da-c42f-4909-883f-7a975ee12abb None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:08.719958 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d04f3-5b28-4744-adcc-e09f83ee203e None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:08.752195 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:08.765947 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ce1924-053d-49bb-b3c9-969ff50b0bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:08.766822 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce1924-053d-49bb-b3c9-969ff50b0bfa None admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:08.767261 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce1924-053d-49bb-b3c9-969ff50b0bfa None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:08.767718 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce1924-053d-49bb-b3c9-969ff50b0bfa None admin] PATH_INFO: `/v3/endpoints` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:08.768594 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ce1924-053d-49bb-b3c9-969ff50b0bfa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:08.770124 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ce1924-053d-49bb-b3c9-969ff50b0bfa None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:08.787721 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b268da-c42f-4909-883f-7a975ee12abb None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:08.795312 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 454/917] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:39:08 2022] POST /identity/v3/endpoints => generated 346 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:39:08.839241 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c49dec5-8f8d-49e7-8437-4893a156ed1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:08.840093 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49dec5-8f8d-49e7-8437-4893a156ed1a None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:08.840469 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49dec5-8f8d-49e7-8437-4893a156ed1a None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:08.840840 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49dec5-8f8d-49e7-8437-4893a156ed1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:08.841862 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c49dec5-8f8d-49e7-8437-4893a156ed1a None admin] RBAC: Adding query filter params (role.id=3655f7c95a7643d296b2c6a9cca87b4b, scope.project.id=644278e7e8934b37a1e7ab20d35764cd, user.id=30fc532f464843a3ad35e4d3c387bb3a) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:08.842274 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c49dec5-8f8d-49e7-8437-4893a156ed1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:08.843989 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c49dec5-8f8d-49e7-8437-4893a156ed1a None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:08.856010 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 464/918] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:39:08 2022] GET /identity/v3/role_assignments?user.id=30fc532f464843a3ad35e4d3c387bb3a&scope.project.id=644278e7e8934b37a1e7ab20d35764cd&role.id=3655f7c95a7643d296b2c6a9cca87b4b => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:09.295233 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394af43-0faf-4598-87e8-8e524f67a908 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:09.295609 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394af43-0faf-4598-87e8-8e524f67a908 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:09.295812 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394af43-0faf-4598-87e8-8e524f67a908 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:09.297469 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 455/919] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:39:09.303311 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a5773-002a-400b-9b20-d3fe45349c2e None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:09.303594 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a5773-002a-400b-9b20-d3fe45349c2e None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:09.303887 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a5773-002a-400b-9b20-d3fe45349c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:09.328115 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-b04a5773-002a-400b-9b20-d3fe45349c2e None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:09.329686 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-b04a5773-002a-400b-9b20-d3fe45349c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:09.380059 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:09.411745 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 465/920] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:39:09.430293 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e394af43-0faf-4598-87e8-8e524f67a908 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:09.540858 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e394af43-0faf-4598-87e8-8e524f67a908 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:09.576979 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda4e6e-3783-4f55-afaf-346ea26eec45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:09.578037 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda4e6e-3783-4f55-afaf-346ea26eec45 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:09.578538 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda4e6e-3783-4f55-afaf-346ea26eec45 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:09.579016 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda4e6e-3783-4f55-afaf-346ea26eec45 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:09.579899 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda4e6e-3783-4f55-afaf-346ea26eec45 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:09.581485 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda4e6e-3783-4f55-afaf-346ea26eec45 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:09.591461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-eeda4e6e-3783-4f55-afaf-346ea26eec45 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 12:39:09.591461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:09.591461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 12:39:09.591461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 12:39:09.591461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 12:39:09.591461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 12:39:09.591461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 12:39:09.591461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:39:09.591461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 12:39:09.591461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:09.591461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 12:39:09.591461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:09.591461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:09.591461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:09.591461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:09.591461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:09.591461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:09.592570 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:09.592570 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:09.592570 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:09.592570 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:09.592570 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:09.592570 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:09.592570 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 12:39:09.592570 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 12:39:09.592570 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 12:39:09.592570 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 12:39:09.592570 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:09.592570 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:09.592570 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:09.592570 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:09.592570 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:09.592570 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:09.593430 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:09.593430 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:09.593430 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:09.593430 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:09.593430 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:09.593430 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:09.593430 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:09.593430 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:09.593430 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:09.593430 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:09.593430 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 12:39:09.593430 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:39:09.593430 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 12:39:09.593430 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:09.599735 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 456/921] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Jul 26 12:39:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:39:09.603440 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04a5773-002a-400b-9b20-d3fe45349c2e None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:09.617542 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1c3b4-ae6f-4328-8fac-4eb68c7599ef None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:09.617863 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1c3b4-ae6f-4328-8fac-4eb68c7599ef None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:09.618093 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1c3b4-ae6f-4328-8fac-4eb68c7599ef None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:09.619818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 457/922] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:39:09.624961 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc8452a-6b69-43ae-b655-b9bf03ece6ce None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:09.625225 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc8452a-6b69-43ae-b655-b9bf03ece6ce None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:09.625511 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc8452a-6b69-43ae-b655-b9bf03ece6ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:09.645514 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-adc8452a-6b69-43ae-b655-b9bf03ece6ce None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:09.646782 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-adc8452a-6b69-43ae-b655-b9bf03ece6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:09.713527 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04a5773-002a-400b-9b20-d3fe45349c2e None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:09.722678 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:09.726204 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 458/923] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:39:09.745545 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc8452a-6b69-43ae-b655-b9bf03ece6ce None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:09.752736 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d4abb7-78aa-4818-8a02-879c083ff62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:09.753650 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4abb7-78aa-4818-8a02-879c083ff62f None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:09.754165 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4abb7-78aa-4818-8a02-879c083ff62f None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:09.754646 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4abb7-78aa-4818-8a02-879c083ff62f None admin] PATH_INFO: `/v3/services` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:09.756163 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d4abb7-78aa-4818-8a02-879c083ff62f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:09.756684 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d4abb7-78aa-4818-8a02-879c083ff62f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:09.758297 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d4abb7-78aa-4818-8a02-879c083ff62f None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:09.768684 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 466/924] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Jul 26 12:39:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:09.770458 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d4abb7-78aa-4818-8a02-879c083ff62f None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:09.826638 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:09.865784 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc8452a-6b69-43ae-b655-b9bf03ece6ce None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:09.886156 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d4abb7-78aa-4818-8a02-879c083ff62f None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:09.903064 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55afd11b-4369-4c44-9192-46abb52e4c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:09.903754 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55afd11b-4369-4c44-9192-46abb52e4c8e None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:09.904040 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55afd11b-4369-4c44-9192-46abb52e4c8e None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:09.904314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55afd11b-4369-4c44-9192-46abb52e4c8e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:09.905026 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55afd11b-4369-4c44-9192-46abb52e4c8e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:09.906344 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55afd11b-4369-4c44-9192-46abb52e4c8e None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:09.916461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-55afd11b-4369-4c44-9192-46abb52e4c8e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 12:39:09.916461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:09.916461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 12:39:09.916461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 12:39:09.916461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 12:39:09.916461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 12:39:09.916461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 12:39:09.916461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:39:09.916461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 12:39:09.916461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:09.916461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 12:39:09.916461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:09.916461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:09.916461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:09.916461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:09.916461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:09.917558 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:09.917558 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:09.917558 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:09.917558 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:09.917558 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:09.917558 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:09.917558 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:09.917558 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 12:39:09.917558 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 12:39:09.917558 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 12:39:09.917558 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 12:39:09.917558 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:09.917558 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:09.917558 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:09.917558 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:09.917558 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:09.918390 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:09.918390 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:09.918390 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:09.918390 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:09.918390 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:09.918390 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:09.918390 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:09.918390 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:09.918390 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:09.918390 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:09.918390 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:09.918390 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 12:39:09.918390 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:39:09.918390 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 12:39:09.918390 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:09.922851 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 459/925] 10.222.0.20 () {56 vars in 1132 bytes} [Tue Jul 26 12:39:09 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:39:09.923803 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55afd11b-4369-4c44-9192-46abb52e4c8e None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:09.932039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df98c6b9-973a-4734-8d96-21726c0fbfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:09.933013 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98c6b9-973a-4734-8d96-21726c0fbfc2 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:09.933557 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98c6b9-973a-4734-8d96-21726c0fbfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:09.934036 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98c6b9-973a-4734-8d96-21726c0fbfc2 None admin] PATH_INFO: `/v3/services` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:09.935032 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df98c6b9-973a-4734-8d96-21726c0fbfc2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:09.935542 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df98c6b9-973a-4734-8d96-21726c0fbfc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:09.937525 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df98c6b9-973a-4734-8d96-21726c0fbfc2 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:09.949151 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 467/926] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Jul 26 12:39:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:39:09.950811 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df98c6b9-973a-4734-8d96-21726c0fbfc2 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:09.980035 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:10.036987 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55afd11b-4369-4c44-9192-46abb52e4c8e None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:10.064883 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df98c6b9-973a-4734-8d96-21726c0fbfc2 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:10.074300 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e8799e-eaa9-4231-b61a-4db3b37e5cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:10.074925 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e8799e-eaa9-4231-b61a-4db3b37e5cf6 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:10.075205 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e8799e-eaa9-4231-b61a-4db3b37e5cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:10.075477 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e8799e-eaa9-4231-b61a-4db3b37e5cf6 None admin] PATH_INFO: `/v3/services` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:10.076195 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e8799e-eaa9-4231-b61a-4db3b37e5cf6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:10.076499 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e8799e-eaa9-4231-b61a-4db3b37e5cf6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:10.077220 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:10.079186 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e8799e-eaa9-4231-b61a-4db3b37e5cf6 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:10.089727 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 460/927] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Jul 26 12:39:09 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:10.090806 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e8799e-eaa9-4231-b61a-4db3b37e5cf6 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:10.104862 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b2c7d-71ef-4237-841a-f7f881825fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:10.105779 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b2c7d-71ef-4237-841a-f7f881825fd4 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:10.106107 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b2c7d-71ef-4237-841a-f7f881825fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:10.106486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b2c7d-71ef-4237-841a-f7f881825fd4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:10.107419 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b2c7d-71ef-4237-841a-f7f881825fd4 None admin] RBAC: Adding query filter params (interface=public, service_id=815fa546088d4a9b92b140a4ee327568, region_id=RegionOne) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:10.107866 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b2c7d-71ef-4237-841a-f7f881825fd4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:10.109381 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b2c7d-71ef-4237-841a-f7f881825fd4 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:10.120426 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 468/928] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Jul 26 12:39:09 2022] GET /identity/v3/endpoints?service_id=815fa546088d4a9b92b140a4ee327568&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:39:10.121703 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b2c7d-71ef-4237-841a-f7f881825fd4 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:10.195674 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e8799e-eaa9-4231-b61a-4db3b37e5cf6 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:10.197581 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:10.232841 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993e7386-9ca2-48c5-b10f-320d3cce978f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:10.233739 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e7386-9ca2-48c5-b10f-320d3cce978f None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:10.234168 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e7386-9ca2-48c5-b10f-320d3cce978f None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:10.234602 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e7386-9ca2-48c5-b10f-320d3cce978f None admin] PATH_INFO: `/v3/services` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:10.235434 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993e7386-9ca2-48c5-b10f-320d3cce978f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:10.235913 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993e7386-9ca2-48c5-b10f-320d3cce978f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:10.237521 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993e7386-9ca2-48c5-b10f-320d3cce978f None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:10.248050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b2c7d-71ef-4237-841a-f7f881825fd4 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:10.248398 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 461/929] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Jul 26 12:39:10 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:10.249484 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993e7386-9ca2-48c5-b10f-320d3cce978f None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:10.288449 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9fb85b-851e-42db-ace5-509ebd0e0e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:10.289506 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9fb85b-851e-42db-ace5-509ebd0e0e9a None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:10.289798 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9fb85b-851e-42db-ace5-509ebd0e0e9a None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:10.290336 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9fb85b-851e-42db-ace5-509ebd0e0e9a None admin] PATH_INFO: `/v3/services` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:10.291023 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9fb85b-851e-42db-ace5-509ebd0e0e9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:10.293454 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9fb85b-851e-42db-ace5-509ebd0e0e9a None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:10.303758 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 469/930] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Jul 26 12:39:10 2022] GET /identity/v3/services => generated 1497 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 12:39:10.379337 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993e7386-9ca2-48c5-b10f-320d3cce978f None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:10.428850 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88259f38-6c6d-48a4-80dc-8ead2598a1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:10.428850 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88259f38-6c6d-48a4-80dc-8ead2598a1de None admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:10.428850 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88259f38-6c6d-48a4-80dc-8ead2598a1de None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:10.428850 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88259f38-6c6d-48a4-80dc-8ead2598a1de None admin] PATH_INFO: `/v3/endpoints` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:10.429266 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88259f38-6c6d-48a4-80dc-8ead2598a1de None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:10.429266 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88259f38-6c6d-48a4-80dc-8ead2598a1de None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:10.452625 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 462/931] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:39:10 2022] POST /identity/v3/endpoints => generated 347 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:39:10.931075 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6be172-6d31-40fd-b078-3d314ecf6583 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:10.931595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6be172-6d31-40fd-b078-3d314ecf6583 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:10.931595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6be172-6d31-40fd-b078-3d314ecf6583 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:10.933301 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 470/932] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:39:10.938832 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d03cd-7c87-4bc5-9988-0973e45d34e4 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:10.939127 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d03cd-7c87-4bc5-9988-0973e45d34e4 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:10.939528 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d03cd-7c87-4bc5-9988-0973e45d34e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:10.944170 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c31e2d-34d8-49cb-986e-ca06610ee250 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:10.944453 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c31e2d-34d8-49cb-986e-ca06610ee250 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:10.944728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c31e2d-34d8-49cb-986e-ca06610ee250 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:10.946321 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 471/933] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 12:39:10.956716 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49293335-d606-4783-a611-fd514b976f84 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:10.957251 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49293335-d606-4783-a611-fd514b976f84 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:10.957880 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49293335-d606-4783-a611-fd514b976f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:10.961856 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-005d03cd-7c87-4bc5-9988-0973e45d34e4 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:10.963243 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-005d03cd-7c87-4bc5-9988-0973e45d34e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:10.980921 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-49293335-d606-4783-a611-fd514b976f84 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:10.982416 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-49293335-d606-4783-a611-fd514b976f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:10.982684 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:11.023266 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:11.045646 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 463/934] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:39:11.073624 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005d03cd-7c87-4bc5-9988-0973e45d34e4 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:11.083996 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 472/935] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 12:39:11.106566 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49293335-d606-4783-a611-fd514b976f84 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:11.196419 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005d03cd-7c87-4bc5-9988-0973e45d34e4 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:11.222971 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49293335-d606-4783-a611-fd514b976f84 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:11.233833 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e71a55-96cf-4c8b-8d11-dec894aab91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:11.234458 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e71a55-96cf-4c8b-8d11-dec894aab91d None admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:11.234734 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e71a55-96cf-4c8b-8d11-dec894aab91d None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:11.234999 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e71a55-96cf-4c8b-8d11-dec894aab91d None admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:11.235823 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e71a55-96cf-4c8b-8d11-dec894aab91d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:11.237256 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e71a55-96cf-4c8b-8d11-dec894aab91d None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:11.241047 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: WARNING keystone.server.flask.common [None req-b5e71a55-96cf-4c8b-8d11-dec894aab91d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 26 12:39:11.261150 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.sql.core [None req-b5e71a55-96cf-4c8b-8d11-dec894aab91d None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 26 12:39:11.261150 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 26 12:39:11.261150 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [parameters: {'id': '62d1e524cbaf4d2daf0d61bd8ae984d3', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 26 12:39:11.261150 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91749) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 26 12:39:11.262069 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608f9383-1aa2-4f4c-b8ca-f12b421bc140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:11.262816 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f9383-1aa2-4f4c-b8ca-f12b421bc140 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:11.263107 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f9383-1aa2-4f4c-b8ca-f12b421bc140 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:11.263432 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f9383-1aa2-4f4c-b8ca-f12b421bc140 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:11.275349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608f9383-1aa2-4f4c-b8ca-f12b421bc140 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:39:11.279470 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-b5e71a55-96cf-4c8b-8d11-dec894aab91d None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 62d1e524cbaf4d2daf0d61bd8ae984d3.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 62d1e524cbaf4d2daf0d61bd8ae984d3. Jul 26 12:39:11.279470 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:11.279470 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 12:39:11.279470 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 12:39:11.279470 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 12:39:11.279470 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 12:39:11.279470 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 12:39:11.279470 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 12:39:11.279470 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 12:39:11.279470 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application conn.query(q) Jul 26 12:39:11.279470 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 12:39:11.279470 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 12:39:11.279470 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 12:39:11.281250 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application result.read() Jul 26 12:39:11.281250 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 12:39:11.281250 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 12:39:11.281250 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 12:39:11.281250 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 12:39:11.281250 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 12:39:11.281250 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 12:39:11.281250 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 12:39:11.281250 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 12:39:11.281250 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 26 12:39:11.281250 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:11.281250 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 26 12:39:11.281250 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:11.281250 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:11.281250 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 26 12:39:11.281250 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 26 12:39:11.281250 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 26 12:39:11.281250 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 26 12:39:11.283243 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 12:39:11.283243 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application next(self.gen) Jul 26 12:39:11.283243 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 26 12:39:11.283243 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application yield resource Jul 26 12:39:11.283243 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 12:39:11.283243 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application next(self.gen) Jul 26 12:39:11.283243 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 26 12:39:11.283243 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self.session.rollback() Jul 26 12:39:11.283243 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 26 12:39:11.283243 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self.force_reraise() Jul 26 12:39:11.283243 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 26 12:39:11.283243 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise self.value Jul 26 12:39:11.283243 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 26 12:39:11.283243 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 26 12:39:11.283243 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 26 12:39:11.283243 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application session.commit() Jul 26 12:39:11.283243 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 26 12:39:11.284629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 26 12:39:11.284629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 26 12:39:11.284629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self._prepare_impl() Jul 26 12:39:11.284629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 26 12:39:11.284629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self.session.flush() Jul 26 12:39:11.284629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 26 12:39:11.284629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self._flush(objects) Jul 26 12:39:11.284629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 26 12:39:11.284629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 26 12:39:11.284629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 26 12:39:11.284629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application compat.raise_( Jul 26 12:39:11.284629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 12:39:11.284629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception Jul 26 12:39:11.284629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 26 12:39:11.284629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application flush_context.execute() Jul 26 12:39:11.284629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 26 12:39:11.284629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rec.execute(self) Jul 26 12:39:11.285844 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 26 12:39:11.285844 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 26 12:39:11.285844 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 26 12:39:11.285844 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 26 12:39:11.285844 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 26 12:39:11.285844 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 26 12:39:11.285844 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 26 12:39:11.285844 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 26 12:39:11.285844 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 26 12:39:11.285844 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 26 12:39:11.285844 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 26 12:39:11.285844 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 26 12:39:11.285844 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 26 12:39:11.285844 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 26 12:39:11.285844 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 26 12:39:11.296516 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 26 12:39:11.296516 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 12:39:11.296516 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception Jul 26 12:39:11.296516 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 12:39:11.296516 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 12:39:11.296516 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 12:39:11.296516 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 12:39:11.296516 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 12:39:11.296516 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 12:39:11.296516 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 12:39:11.296516 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application conn.query(q) Jul 26 12:39:11.296516 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 12:39:11.296516 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 12:39:11.296516 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 12:39:11.296516 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application result.read() Jul 26 12:39:11.296516 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 12:39:11.296516 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 12:39:11.298402 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 12:39:11.298402 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 12:39:11.298402 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 12:39:11.298402 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 12:39:11.298402 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 12:39:11.298402 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 12:39:11.298402 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 26 12:39:11.298402 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 26 12:39:11.298402 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [parameters: {'id': '62d1e524cbaf4d2daf0d61bd8ae984d3', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 26 12:39:11.298402 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 26 12:39:11.298402 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:11.298402 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 12:39:11.298402 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:11.298402 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:11.298402 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 26 12:39:11.300117 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 26 12:39:11.300117 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 26 12:39:11.300117 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 26 12:39:11.300117 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 26 12:39:11.300117 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:11.300117 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 12:39:11.300117 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:11.300117 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:11.300117 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:11.300117 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:11.300117 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:11.300117 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:11.300117 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:11.300117 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:11.300117 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:11.300117 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:11.300117 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:11.301890 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:11.301890 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 26 12:39:11.301890 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 26 12:39:11.301890 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:11.301890 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:11.301890 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 26 12:39:11.301890 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 26 12:39:11.301890 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 62d1e524cbaf4d2daf0d61bd8ae984d3. Jul 26 12:39:11.301890 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:11.301890 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 464/936] 10.222.0.20 () {60 vars in 1140 bytes} [Tue Jul 26 12:39:11 2022] POST /identity/v3/projects => generated 262 bytes in 208 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:39:11.303046 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-608f9383-1aa2-4f4c-b8ca-f12b421bc140 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 12:39:11.303046 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:11.303046 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:11.303046 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:11.303046 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:11.303046 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:11.303046 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:11.303046 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:11.303046 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:11.303046 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:11.303046 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:11.303046 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:11.303046 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:39:11.303046 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:39:11.303046 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:39:11.303046 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:39:11.304820 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:39:11.304820 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:39:11.304820 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:39:11.304820 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:39:11.304820 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:11.304820 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:11.304820 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:11.304820 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:11.304820 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:11.304820 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:11.304820 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:11.304820 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:11.304820 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:11.304820 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:11.304820 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:11.304820 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:11.317153 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:11.317153 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:11.317153 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:11.317153 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:11.317153 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:39:11.317153 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:39:11.317153 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:39:11.317153 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:39:11.317153 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:39:11.317153 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:39:11.317153 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 12:39:11.317153 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:11.318997 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e71a55-96cf-4c8b-8d11-dec894aab91d None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:11.319127 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 473/937] 10.222.0.20 () {56 vars in 1120 bytes} [Tue Jul 26 12:39:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:39:11.319258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:11.319382 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608f9383-1aa2-4f4c-b8ca-f12b421bc140 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:11.329776 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:11.407462 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e71a55-96cf-4c8b-8d11-dec894aab91d None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:11.416007 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608f9383-1aa2-4f4c-b8ca-f12b421bc140 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:11.445631 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f771a4-ecff-421f-9eb2-08a0a80c1fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:11.446300 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f771a4-ecff-421f-9eb2-08a0a80c1fbd None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:11.446750 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f771a4-ecff-421f-9eb2-08a0a80c1fbd None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:11.447087 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f771a4-ecff-421f-9eb2-08a0a80c1fbd None admin] PATH_INFO: `/v3/projects/service` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:11.454997 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f771a4-ecff-421f-9eb2-08a0a80c1fbd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:11.455911 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdfe9c1-3cc6-461b-957c-d81358bf74fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:11.456536 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfe9c1-3cc6-461b-957c-d81358bf74fa None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:11.456819 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfe9c1-3cc6-461b-957c-d81358bf74fa None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:11.457055 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f771a4-ecff-421f-9eb2-08a0a80c1fbd None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:11.457211 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfe9c1-3cc6-461b-957c-d81358bf74fa None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:11.457863 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdfe9c1-3cc6-461b-957c-d81358bf74fa None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:11.458174 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdfe9c1-3cc6-461b-957c-d81358bf74fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:11.460439 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdfe9c1-3cc6-461b-957c-d81358bf74fa None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:11.471425 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-32f771a4-ecff-421f-9eb2-08a0a80c1fbd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:39:11.471425 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:11.471425 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:11.471425 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:11.471425 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:11.471425 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:11.471425 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:11.471425 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:11.471425 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:11.471425 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:11.471425 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:11.471425 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:11.471425 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 12:39:11.471425 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 12:39:11.471425 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 12:39:11.473251 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 12:39:11.473251 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:11.473251 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:11.473251 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:11.473251 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:11.473251 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:11.473251 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:11.473251 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:11.473251 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:11.473251 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:11.473251 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:11.473251 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:11.473251 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:11.473251 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:11.473251 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:11.473251 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:11.473251 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:11.475343 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 12:39:11.475343 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 12:39:11.475343 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 12:39:11.475343 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 12:39:11.475343 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 12:39:11.475343 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 12:39:11.475343 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:39:11.475343 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:11.476337 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 474/938] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Jul 26 12:39:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:39:11.483561 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 465/939] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:39:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:39:11.483751 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdfe9c1-3cc6-461b-957c-d81358bf74fa None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:11.485768 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f771a4-ecff-421f-9eb2-08a0a80c1fbd None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:11.553833 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:11.584350 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:11.598530 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f771a4-ecff-421f-9eb2-08a0a80c1fbd None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:11.599225 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdfe9c1-3cc6-461b-957c-d81358bf74fa None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:11.641080 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21b78dd-48c6-4a9c-802a-1f5fc3ef6772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:11.641703 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b78dd-48c6-4a9c-802a-1f5fc3ef6772 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:11.641980 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b78dd-48c6-4a9c-802a-1f5fc3ef6772 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:11.642249 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b78dd-48c6-4a9c-802a-1f5fc3ef6772 None admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:11.642969 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21b78dd-48c6-4a9c-802a-1f5fc3ef6772 None admin] RBAC: Adding query filter params (name=service) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:11.643268 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21b78dd-48c6-4a9c-802a-1f5fc3ef6772 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:11.644775 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21b78dd-48c6-4a9c-802a-1f5fc3ef6772 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:11.645664 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d4e24e-d78a-46a9-bf47-bf4883734bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:11.647469 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4e24e-d78a-46a9-bf47-bf4883734bb3 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:11.648190 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4e24e-d78a-46a9-bf47-bf4883734bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:11.648732 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4e24e-d78a-46a9-bf47-bf4883734bb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:11.667160 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d4e24e-d78a-46a9-bf47-bf4883734bb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:11.669120 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d4e24e-d78a-46a9-bf47-bf4883734bb3 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:11.689968 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 466/940] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Jul 26 12:39:11 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:39:11.695953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-91d4e24e-d78a-46a9-bf47-bf4883734bb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:39:11.695953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:11.695953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:11.695953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:11.695953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:11.695953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:11.695953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:11.695953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:11.695953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:11.695953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:11.695953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:11.695953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:11.695953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:39:11.695953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:39:11.695953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:39:11.696809 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:39:11.696809 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:11.696809 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:11.696809 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:11.696809 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:11.696809 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:11.696809 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:11.696809 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:11.696809 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:11.696809 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:11.696809 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:11.696809 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:11.696809 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:11.696809 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:11.696809 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:11.696809 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:11.696809 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:11.697708 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:39:11.697708 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:39:11.697708 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:39:11.697708 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:11.708154 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 475/941] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:39:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:39:11.710090 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21b78dd-48c6-4a9c-802a-1f5fc3ef6772 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:11.916216 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21b78dd-48c6-4a9c-802a-1f5fc3ef6772 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:11.941225 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:11.952826 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea573d40-5daa-4220-8b36-266d0aa8a0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:11.953375 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea573d40-5daa-4220-8b36-266d0aa8a0df None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:11.953655 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea573d40-5daa-4220-8b36-266d0aa8a0df None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:11.953925 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea573d40-5daa-4220-8b36-266d0aa8a0df None admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:11.954627 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea573d40-5daa-4220-8b36-266d0aa8a0df None admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:11.954927 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea573d40-5daa-4220-8b36-266d0aa8a0df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:11.956324 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea573d40-5daa-4220-8b36-266d0aa8a0df None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:11.969056 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 467/942] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:39:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:11.975467 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d4e24e-d78a-46a9-bf47-bf4883734bb3 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:12.100534 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d4e24e-d78a-46a9-bf47-bf4883734bb3 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:12.137894 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a279ffc-3bfa-46bc-b68f-c247e5563cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:12.138693 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a279ffc-3bfa-46bc-b68f-c247e5563cc8 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:12.139085 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a279ffc-3bfa-46bc-b68f-c247e5563cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:12.139451 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a279ffc-3bfa-46bc-b68f-c247e5563cc8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:12.149010 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a279ffc-3bfa-46bc-b68f-c247e5563cc8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:12.150667 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a279ffc-3bfa-46bc-b68f-c247e5563cc8 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:12.156068 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:12.166044 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-7a279ffc-3bfa-46bc-b68f-c247e5563cc8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 12:39:12.166044 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:12.166044 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:12.166044 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:12.166044 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:12.166044 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:12.166044 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:12.166044 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:12.166044 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:12.166044 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:12.166044 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:12.166044 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:12.166044 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 12:39:12.166044 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 12:39:12.166044 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 12:39:12.166044 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 12:39:12.167050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:12.167050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:12.167050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 12:39:12.167050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:39:12.167050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 12:39:12.167050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:39:12.167050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:12.167050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:12.167050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:12.167050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:12.167050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:12.167050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:12.167050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:12.167050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:12.167050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:12.167050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:12.167050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:12.167050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:12.168030 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:12.168030 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:12.168030 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 12:39:12.168030 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 12:39:12.168030 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 12:39:12.168030 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 12:39:12.168030 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 12:39:12.168030 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 12:39:12.168030 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 12:39:12.168030 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:12.168030 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 476/943] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Jul 26 12:39:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:39:12.176861 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea573d40-5daa-4220-8b36-266d0aa8a0df None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:12.328578 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea573d40-5daa-4220-8b36-266d0aa8a0df None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:12.364130 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ad5a5-80cd-4f8e-801b-bdba4a9cac4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:12.365627 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ad5a5-80cd-4f8e-801b-bdba4a9cac4c None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:12.365789 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ad5a5-80cd-4f8e-801b-bdba4a9cac4c None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:12.366057 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ad5a5-80cd-4f8e-801b-bdba4a9cac4c None admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:12.366838 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412ad5a5-80cd-4f8e-801b-bdba4a9cac4c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:12.367141 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412ad5a5-80cd-4f8e-801b-bdba4a9cac4c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:12.368584 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412ad5a5-80cd-4f8e-801b-bdba4a9cac4c None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:12.387715 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 468/944] 10.222.0.20 () {56 vars in 1149 bytes} [Tue Jul 26 12:39:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:12.393317 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a279ffc-3bfa-46bc-b68f-c247e5563cc8 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:12.510290 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a279ffc-3bfa-46bc-b68f-c247e5563cc8 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:12.544581 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc652a0-d528-47b2-879f-40fe125e64ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:12.545231 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc652a0-d528-47b2-879f-40fe125e64ed None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:12.545513 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc652a0-d528-47b2-879f-40fe125e64ed None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:12.545787 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc652a0-d528-47b2-879f-40fe125e64ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:12.557940 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc652a0-d528-47b2-879f-40fe125e64ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:12.559412 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc652a0-d528-47b2-879f-40fe125e64ed None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:12.568035 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc89b1-58f2-4479-a549-3bda28e433a4 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:12.568540 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc89b1-58f2-4479-a549-3bda28e433a4 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:12.568631 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc89b1-58f2-4479-a549-3bda28e433a4 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:12.570183 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 469/945] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:39:12.575137 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7cdad-af01-41c6-8c1f-2c35ca32a36b None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:12.575417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7cdad-af01-41c6-8c1f-2c35ca32a36b None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:12.575523 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-bbc652a0-d528-47b2-879f-40fe125e64ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:39:12.575523 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:12.575523 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:12.575523 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:12.575523 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:12.575523 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:12.575523 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:12.575523 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:12.575523 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:12.575523 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:12.575523 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:12.575523 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:12.575523 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:39:12.575523 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:39:12.575523 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:39:12.576390 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:39:12.576390 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:12.576390 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:12.576390 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:12.576390 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:12.576390 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:12.576390 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:12.576390 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:12.576390 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:12.576390 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:12.576390 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:12.576390 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:12.576390 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:12.576390 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:12.576390 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:12.576390 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:12.576390 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:12.577277 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:39:12.577277 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:39:12.577277 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:39:12.577277 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:12.577507 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7cdad-af01-41c6-8c1f-2c35ca32a36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:12.577577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 477/946] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:39:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:39:12.583697 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc652a0-d528-47b2-879f-40fe125e64ed None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:12.600831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-d2b7cdad-af01-41c6-8c1f-2c35ca32a36b None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:12.602722 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-d2b7cdad-af01-41c6-8c1f-2c35ca32a36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:12.640976 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:12.649727 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:12.676998 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 470/947] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 12:39:12.678785 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32eddf0-13f4-4b6b-af63-981e462dc167 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:12.679080 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32eddf0-13f4-4b6b-af63-981e462dc167 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:12.679359 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32eddf0-13f4-4b6b-af63-981e462dc167 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:12.680970 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 471/948] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:39:12.689587 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f899b2-9aef-4be6-8903-06d22ec33764 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:12.690161 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f899b2-9aef-4be6-8903-06d22ec33764 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:12.690635 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f899b2-9aef-4be6-8903-06d22ec33764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:12.711130 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc652a0-d528-47b2-879f-40fe125e64ed None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:12.711351 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-f8f899b2-9aef-4be6-8903-06d22ec33764 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:12.712411 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-f8f899b2-9aef-4be6-8903-06d22ec33764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:12.751145 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99f6b3-cf8e-4491-a33a-896d5dd53096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:12.752111 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99f6b3-cf8e-4491-a33a-896d5dd53096 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:12.752631 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99f6b3-cf8e-4491-a33a-896d5dd53096 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:12.753105 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99f6b3-cf8e-4491-a33a-896d5dd53096 None admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:12.753981 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a99f6b3-cf8e-4491-a33a-896d5dd53096 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:12.754469 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a99f6b3-cf8e-4491-a33a-896d5dd53096 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:12.756153 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a99f6b3-cf8e-4491-a33a-896d5dd53096 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:12.770575 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 478/949] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:39:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:39:12.771872 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99f6b3-cf8e-4491-a33a-896d5dd53096 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:12.787539 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 472/950] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:39:12.788700 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f899b2-9aef-4be6-8903-06d22ec33764 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:12.917532 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f899b2-9aef-4be6-8903-06d22ec33764 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:12.918187 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99f6b3-cf8e-4491-a33a-896d5dd53096 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:12.919357 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:12.933074 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:12.956025 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c046cd-9499-4a81-b04d-df61beac0c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:12.956668 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c046cd-9499-4a81-b04d-df61beac0c08 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:12.956948 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c046cd-9499-4a81-b04d-df61beac0c08 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:12.957271 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c046cd-9499-4a81-b04d-df61beac0c08 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:12.958462 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0aa666-86a6-46b2-8236-7535ff929fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:12.959107 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0aa666-86a6-46b2-8236-7535ff929fbb None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:12.959383 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0aa666-86a6-46b2-8236-7535ff929fbb None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:12.959674 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0aa666-86a6-46b2-8236-7535ff929fbb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:12.960313 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0aa666-86a6-46b2-8236-7535ff929fbb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:12.961684 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0aa666-86a6-46b2-8236-7535ff929fbb None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:12.964607 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c046cd-9499-4a81-b04d-df61beac0c08 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:12.966267 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c046cd-9499-4a81-b04d-df61beac0c08 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:12.970758 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-7c0aa666-86a6-46b2-8236-7535ff929fbb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 12:39:12.970758 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:12.970758 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 12:39:12.970758 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 12:39:12.970758 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 12:39:12.970758 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 12:39:12.970758 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 12:39:12.970758 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:39:12.970758 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 12:39:12.970758 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:12.970758 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 12:39:12.970758 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:12.970758 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:12.970758 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:12.970758 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:12.970758 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:12.970758 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:12.971783 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:12.971783 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:12.971783 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:12.971783 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:12.971783 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:12.971783 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:12.971783 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 12:39:12.971783 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 12:39:12.971783 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 12:39:12.971783 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 12:39:12.971783 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:12.971783 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:12.971783 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:12.971783 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:12.971783 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:12.971783 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:12.972642 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:12.972642 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:12.972642 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:12.972642 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:12.972642 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:12.972642 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:12.972642 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:12.972642 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:12.972642 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:12.972642 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:12.972642 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 12:39:12.972642 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:39:12.972642 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 12:39:12.972642 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:12.972642 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 479/951] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Jul 26 12:39:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:39:12.973426 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0aa666-86a6-46b2-8236-7535ff929fbb None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:12.982858 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-06c046cd-9499-4a81-b04d-df61beac0c08 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:39:12.982858 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:12.982858 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:12.982858 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:12.982858 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:12.982858 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:12.982858 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:12.982858 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:12.982858 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:12.982858 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:12.982858 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:12.982858 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:12.982858 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 12:39:12.982858 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 12:39:12.982858 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 12:39:12.983802 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 12:39:12.983802 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:12.983802 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:12.983802 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:12.983802 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:12.983802 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:12.983802 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:12.983802 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:12.983802 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:12.983802 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:12.983802 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:12.983802 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:12.983802 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:12.983802 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:12.983802 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:12.983802 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:12.983802 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:12.990909 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 12:39:12.990909 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 12:39:12.990909 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 12:39:12.990909 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 12:39:12.990909 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 12:39:12.990909 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 12:39:12.990909 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:39:12.990909 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:12.990909 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 473/952] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:39:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:39:12.990909 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c046cd-9499-4a81-b04d-df61beac0c08 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:13.088636 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0aa666-86a6-46b2-8236-7535ff929fbb None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:13.092236 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c046cd-9499-4a81-b04d-df61beac0c08 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:13.127578 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de3c6bd-9c61-4a25-b431-7ac82857e0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:13.128246 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3c6bd-9c61-4a25-b431-7ac82857e0ac None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:13.128527 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3c6bd-9c61-4a25-b431-7ac82857e0ac None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:13.128794 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3c6bd-9c61-4a25-b431-7ac82857e0ac None admin] PATH_INFO: `/v3/services/compute` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:13.129409 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de3c6bd-9c61-4a25-b431-7ac82857e0ac None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:13.130221 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779facc-6fb1-4510-95fd-34e1493b0e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:13.131390 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de3c6bd-9c61-4a25-b431-7ac82857e0ac None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:13.132164 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779facc-6fb1-4510-95fd-34e1493b0e29 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:13.132590 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779facc-6fb1-4510-95fd-34e1493b0e29 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:13.133004 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779facc-6fb1-4510-95fd-34e1493b0e29 None admin] PATH_INFO: `/v3/services` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:13.133812 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2779facc-6fb1-4510-95fd-34e1493b0e29 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:13.134309 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2779facc-6fb1-4510-95fd-34e1493b0e29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:13.135693 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2779facc-6fb1-4510-95fd-34e1493b0e29 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:13.140762 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-2de3c6bd-9c61-4a25-b431-7ac82857e0ac None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 12:39:13.140762 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:13.140762 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 12:39:13.140762 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 12:39:13.140762 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 12:39:13.140762 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 12:39:13.140762 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 12:39:13.140762 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:39:13.140762 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 12:39:13.140762 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:13.140762 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 12:39:13.140762 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:13.140762 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:13.140762 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:13.140762 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:13.140762 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:13.140762 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:13.141865 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:13.141865 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:13.141865 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:13.141865 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:13.141865 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:13.141865 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:13.141865 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 12:39:13.141865 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 12:39:13.141865 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 12:39:13.141865 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 12:39:13.141865 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:13.141865 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:13.141865 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:13.141865 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:13.141865 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:13.141865 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:13.147930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:13.147930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:13.147930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:13.147930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:13.147930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:13.147930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:13.147930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:13.147930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:13.147930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:13.147930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:13.147930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 12:39:13.147930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:39:13.147930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 12:39:13.147930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:13.147930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 480/953] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:39:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:39:13.154212 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de3c6bd-9c61-4a25-b431-7ac82857e0ac None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:13.154300 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 474/954] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Jul 26 12:39:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:13.154300 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779facc-6fb1-4510-95fd-34e1493b0e29 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:13.188614 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:13.208079 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:13.272424 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de3c6bd-9c61-4a25-b431-7ac82857e0ac None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:13.272630 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779facc-6fb1-4510-95fd-34e1493b0e29 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:13.308191 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e710f60d-8611-4c91-b6cd-e6c296dfabeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:13.308576 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f453df-d019-4d8b-b0be-85ff5db4f1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:13.308786 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710f60d-8611-4c91-b6cd-e6c296dfabeb None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:13.309067 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710f60d-8611-4c91-b6cd-e6c296dfabeb None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:13.309165 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f453df-d019-4d8b-b0be-85ff5db4f1b1 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:13.309336 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710f60d-8611-4c91-b6cd-e6c296dfabeb None admin] PATH_INFO: `/v3/services` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:13.309442 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f453df-d019-4d8b-b0be-85ff5db4f1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:13.309643 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f453df-d019-4d8b-b0be-85ff5db4f1b1 None admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:13.310019 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e710f60d-8611-4c91-b6cd-e6c296dfabeb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:13.310321 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e710f60d-8611-4c91-b6cd-e6c296dfabeb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:13.310415 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f453df-d019-4d8b-b0be-85ff5db4f1b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:13.310687 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f453df-d019-4d8b-b0be-85ff5db4f1b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:13.311713 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e710f60d-8611-4c91-b6cd-e6c296dfabeb None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:13.312161 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f453df-d019-4d8b-b0be-85ff5db4f1b1 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:13.317153 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 475/955] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Jul 26 12:39:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:13.318805 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e710f60d-8611-4c91-b6cd-e6c296dfabeb None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:13.325220 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 481/956] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 26 12:39:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:13.328572 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f453df-d019-4d8b-b0be-85ff5db4f1b1 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:13.420615 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:13.432277 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:13.433665 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e710f60d-8611-4c91-b6cd-e6c296dfabeb None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:13.435567 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f453df-d019-4d8b-b0be-85ff5db4f1b1 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:13.484170 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87afd87d-8b22-48eb-a00f-5ce0f77a9d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:13.484898 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87afd87d-8b22-48eb-a00f-5ce0f77a9d3d None admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:13.485191 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87afd87d-8b22-48eb-a00f-5ce0f77a9d3d None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:13.485467 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87afd87d-8b22-48eb-a00f-5ce0f77a9d3d None admin] PATH_INFO: `/v3/endpoints` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:13.486192 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87afd87d-8b22-48eb-a00f-5ce0f77a9d3d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:13.487573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87afd87d-8b22-48eb-a00f-5ce0f77a9d3d None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:13.488101 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c30c0d-3d64-413e-9e8b-abf6bed9e386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:13.488715 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c30c0d-3d64-413e-9e8b-abf6bed9e386 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:13.488991 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c30c0d-3d64-413e-9e8b-abf6bed9e386 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:13.489256 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c30c0d-3d64-413e-9e8b-abf6bed9e386 None admin] PATH_INFO: `/v3/services` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:13.489957 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c30c0d-3d64-413e-9e8b-abf6bed9e386 None admin] RBAC: Adding query filter params (name=compute) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:13.490539 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c30c0d-3d64-413e-9e8b-abf6bed9e386 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:13.491875 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c30c0d-3d64-413e-9e8b-abf6bed9e386 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:13.503189 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 476/957] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Jul 26 12:39:13 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:13.503189 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c30c0d-3d64-413e-9e8b-abf6bed9e386 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:13.510680 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 482/958] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:39:13 2022] POST /identity/v3/endpoints => generated 349 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:39:13.511371 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87afd87d-8b22-48eb-a00f-5ce0f77a9d3d None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:13.635982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c30c0d-3d64-413e-9e8b-abf6bed9e386 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:13.674549 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87afd87d-8b22-48eb-a00f-5ce0f77a9d3d None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:13.683964 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce206c-4516-4b4f-ab0a-3db9a4a2e196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:13.684677 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce206c-4516-4b4f-ab0a-3db9a4a2e196 None admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:13.685035 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce206c-4516-4b4f-ab0a-3db9a4a2e196 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:13.685382 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce206c-4516-4b4f-ab0a-3db9a4a2e196 None admin] PATH_INFO: `/v3/projects/644278e7e8934b37a1e7ab20d35764cd/users/30fc532f464843a3ad35e4d3c387bb3a/roles/3655f7c95a7643d296b2c6a9cca87b4b` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:13.688395 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ce206c-4516-4b4f-ab0a-3db9a4a2e196 None admin] RBAC: Authorizing `identity:create_grant(project_id=644278e7e8934b37a1e7ab20d35764cd, user_id=30fc532f464843a3ad35e4d3c387bb3a, role_id=3655f7c95a7643d296b2c6a9cca87b4b)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:13.690377 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ce206c-4516-4b4f-ab0a-3db9a4a2e196 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:13.709657 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 477/959] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 26 12:39:13 2022] PUT /identity/v3/projects/644278e7e8934b37a1e7ab20d35764cd/users/30fc532f464843a3ad35e4d3c387bb3a/roles/3655f7c95a7643d296b2c6a9cca87b4b => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:39:13.721525 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd373b3-5216-4621-91b4-531ebcf70ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:13.722168 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd373b3-5216-4621-91b4-531ebcf70ea0 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:13.722446 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd373b3-5216-4621-91b4-531ebcf70ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:13.722716 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd373b3-5216-4621-91b4-531ebcf70ea0 None admin] PATH_INFO: `/v3/services` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:13.723432 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd373b3-5216-4621-91b4-531ebcf70ea0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:13.723770 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd373b3-5216-4621-91b4-531ebcf70ea0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:13.725981 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd373b3-5216-4621-91b4-531ebcf70ea0 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:13.730854 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 483/960] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Jul 26 12:39:13 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:14.088013 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7c304-fb77-42fc-ae77-05bd3db435ed None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:14.088376 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7c304-fb77-42fc-ae77-05bd3db435ed None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:14.088579 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7c304-fb77-42fc-ae77-05bd3db435ed None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:14.090205 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 478/961] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:39:14.095375 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e498c-4d0c-47fb-b070-6594ed02a90f None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:14.095860 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e498c-4d0c-47fb-b070-6594ed02a90f None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:14.096038 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e498c-4d0c-47fb-b070-6594ed02a90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:14.116438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-ba1e498c-4d0c-47fb-b070-6594ed02a90f None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:14.117654 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-ba1e498c-4d0c-47fb-b070-6594ed02a90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:14.130646 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:14.207429 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 484/962] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:39:14.232398 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb7c304-fb77-42fc-ae77-05bd3db435ed None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:14.251140 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:14.349763 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb7c304-fb77-42fc-ae77-05bd3db435ed None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:14.385082 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140f4e0f-9ef5-4d0a-8bec-0aad4f26db9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:14.385697 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f4e0f-9ef5-4d0a-8bec-0aad4f26db9e None admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:14.385981 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f4e0f-9ef5-4d0a-8bec-0aad4f26db9e None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:14.386259 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f4e0f-9ef5-4d0a-8bec-0aad4f26db9e None admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:14.386949 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140f4e0f-9ef5-4d0a-8bec-0aad4f26db9e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:14.388408 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140f4e0f-9ef5-4d0a-8bec-0aad4f26db9e None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:14.404134 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.sql.core [None req-140f4e0f-9ef5-4d0a-8bec-0aad4f26db9e None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 26 12:39:14.404134 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 26 12:39:14.404134 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [parameters: {'user_id': '14328b195d174ee29257ea96e9ccef0f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 26 12:39:14.404134 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91749) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 26 12:39:14.411896 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-140f4e0f-9ef5-4d0a-8bec-0aad4f26db9e None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 26 12:39:14.411896 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:14.411896 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 12:39:14.411896 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 12:39:14.411896 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 12:39:14.411896 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 12:39:14.411896 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 12:39:14.411896 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 12:39:14.411896 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 12:39:14.411896 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application conn.query(q) Jul 26 12:39:14.411896 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 12:39:14.411896 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 12:39:14.411896 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 12:39:14.411896 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application result.read() Jul 26 12:39:14.411896 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 12:39:14.412779 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 12:39:14.412779 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 12:39:14.412779 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 12:39:14.412779 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 12:39:14.412779 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 12:39:14.412779 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 12:39:14.412779 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 12:39:14.412779 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 26 12:39:14.412779 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:14.412779 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 26 12:39:14.412779 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:14.412779 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:14.412779 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 26 12:39:14.412779 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 26 12:39:14.412779 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 26 12:39:14.412779 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 26 12:39:14.412779 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 12:39:14.412779 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application next(self.gen) Jul 26 12:39:14.413903 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 26 12:39:14.413903 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application yield resource Jul 26 12:39:14.413903 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 12:39:14.413903 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application next(self.gen) Jul 26 12:39:14.413903 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 26 12:39:14.413903 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self.session.rollback() Jul 26 12:39:14.413903 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 26 12:39:14.413903 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self.force_reraise() Jul 26 12:39:14.413903 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 26 12:39:14.413903 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise self.value Jul 26 12:39:14.413903 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 26 12:39:14.413903 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 26 12:39:14.413903 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 26 12:39:14.413903 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application session.commit() Jul 26 12:39:14.413903 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 26 12:39:14.413903 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 26 12:39:14.413903 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 26 12:39:14.415400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self._prepare_impl() Jul 26 12:39:14.415400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 26 12:39:14.415400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self.session.flush() Jul 26 12:39:14.415400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 26 12:39:14.415400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self._flush(objects) Jul 26 12:39:14.415400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 26 12:39:14.415400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 26 12:39:14.415400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 26 12:39:14.415400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application compat.raise_( Jul 26 12:39:14.415400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 12:39:14.415400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception Jul 26 12:39:14.415400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 26 12:39:14.415400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application flush_context.execute() Jul 26 12:39:14.415400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 26 12:39:14.415400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rec.execute(self) Jul 26 12:39:14.415400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 26 12:39:14.415400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 26 12:39:14.416468 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 26 12:39:14.416468 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 26 12:39:14.416468 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 26 12:39:14.416468 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 26 12:39:14.416468 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 26 12:39:14.416468 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 26 12:39:14.416468 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 26 12:39:14.416468 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 26 12:39:14.416468 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 26 12:39:14.416468 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 26 12:39:14.416468 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 26 12:39:14.416468 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 26 12:39:14.416468 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 26 12:39:14.416468 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 26 12:39:14.416468 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 12:39:14.416468 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception Jul 26 12:39:14.417310 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 12:39:14.417310 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 12:39:14.417310 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 12:39:14.417310 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 12:39:14.417310 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 12:39:14.417310 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 12:39:14.417310 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 12:39:14.417310 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application conn.query(q) Jul 26 12:39:14.417310 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 12:39:14.417310 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 12:39:14.417310 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 12:39:14.417310 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application result.read() Jul 26 12:39:14.417310 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 12:39:14.417310 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 12:39:14.417310 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 12:39:14.417310 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 12:39:14.424269 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 12:39:14.424269 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 12:39:14.424269 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 12:39:14.424269 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 12:39:14.424269 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 26 12:39:14.424269 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 26 12:39:14.424269 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [parameters: {'user_id': '14328b195d174ee29257ea96e9ccef0f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 26 12:39:14.424269 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 26 12:39:14.424269 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:14.424269 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 12:39:14.424269 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:14.424269 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:14.424269 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:14.424269 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:14.424269 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:14.425258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:14.425258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:14.425258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:14.425258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:14.425258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:14.425258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:14.425258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:14.425258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 26 12:39:14.425258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 26 12:39:14.425258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:14.425258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:14.425258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 12:39:14.425258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:39:14.425258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 12:39:14.425258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:39:14.425258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 26 12:39:14.425258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 26 12:39:14.426358 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 26 12:39:14.426358 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 26 12:39:14.426358 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 26 12:39:14.426358 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 26 12:39:14.426358 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 26 12:39:14.426358 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:14.426358 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 479/963] 10.222.0.20 () {60 vars in 1131 bytes} [Tue Jul 26 12:39:14 2022] POST /identity/v3/users => generated 160 bytes in 182 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:39:14.426833 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1e498c-4d0c-47fb-b070-6594ed02a90f None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:14.540609 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1e498c-4d0c-47fb-b070-6594ed02a90f None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:14.582067 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a107f80-8bd8-40ad-8824-17296ba93075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:14.582708 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a107f80-8bd8-40ad-8824-17296ba93075 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:14.582989 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a107f80-8bd8-40ad-8824-17296ba93075 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:14.583286 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a107f80-8bd8-40ad-8824-17296ba93075 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:14.590831 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a107f80-8bd8-40ad-8824-17296ba93075 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:14.592313 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a107f80-8bd8-40ad-8824-17296ba93075 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:14.604728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-0a107f80-8bd8-40ad-8824-17296ba93075 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 12:39:14.604728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:14.604728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:14.604728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:14.604728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:14.604728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:14.604728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:14.604728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:14.604728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:14.604728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:14.604728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:14.604728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:14.604728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 12:39:14.604728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 12:39:14.604728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 12:39:14.604728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 12:39:14.606388 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:14.606388 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:14.606388 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 12:39:14.606388 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:39:14.606388 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 12:39:14.606388 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:39:14.606388 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:14.606388 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:14.606388 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:14.606388 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:14.606388 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:14.606388 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:14.606388 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:14.606388 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:14.606388 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:14.606388 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:14.606388 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:14.606388 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:14.607419 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:14.607419 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:14.607419 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 12:39:14.607419 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 12:39:14.607419 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 12:39:14.607419 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 12:39:14.607419 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 12:39:14.607419 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 12:39:14.607419 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 12:39:14.607419 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:14.607419 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 485/964] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:39:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:39:14.611980 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140f4e0f-9ef5-4d0a-8bec-0aad4f26db9e None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:14.693478 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:14.718391 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140f4e0f-9ef5-4d0a-8bec-0aad4f26db9e None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:14.756266 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946e5354-5a51-4915-b412-a596c8890915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:14.756894 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e5354-5a51-4915-b412-a596c8890915 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:14.757174 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e5354-5a51-4915-b412-a596c8890915 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:14.757505 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e5354-5a51-4915-b412-a596c8890915 None admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:14.758281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946e5354-5a51-4915-b412-a596c8890915 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:14.758584 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946e5354-5a51-4915-b412-a596c8890915 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:14.760309 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946e5354-5a51-4915-b412-a596c8890915 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:14.803325 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 480/965] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 26 12:39:14 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:15.655008 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea1339-98d6-46ff-a296-4543579de77b None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:15.655559 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea1339-98d6-46ff-a296-4543579de77b None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:15.655559 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea1339-98d6-46ff-a296-4543579de77b None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:15.657085 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 486/966] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:39:15.662273 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d297d2-e880-4b40-aa44-5293f1bc41c7 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:15.662565 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d297d2-e880-4b40-aa44-5293f1bc41c7 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:15.662844 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d297d2-e880-4b40-aa44-5293f1bc41c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:15.681566 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-29d297d2-e880-4b40-aa44-5293f1bc41c7 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:15.683085 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-29d297d2-e880-4b40-aa44-5293f1bc41c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:15.755624 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 481/967] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:39:15.777547 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea1339-98d6-46ff-a296-4543579de77b None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:15.787842 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:15.817720 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c569ca-e79c-4e6c-9040-ecc0fdd57f88 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:15.818062 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c569ca-e79c-4e6c-9040-ecc0fdd57f88 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:15.818276 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c569ca-e79c-4e6c-9040-ecc0fdd57f88 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:15.819670 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 482/968] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 12:39:15.824876 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b9768-24d3-4570-a9eb-3069020bb5a8 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:15.825135 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b9768-24d3-4570-a9eb-3069020bb5a8 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:15.825406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b9768-24d3-4570-a9eb-3069020bb5a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:15.844194 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-531b9768-24d3-4570-a9eb-3069020bb5a8 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:15.845388 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-531b9768-24d3-4570-a9eb-3069020bb5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:15.897428 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea1339-98d6-46ff-a296-4543579de77b None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:15.906872 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:15.915658 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 483/969] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 12:39:15.917047 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235899ee-6cc2-4854-af82-0dc3d9b64f98 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:15.917314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235899ee-6cc2-4854-af82-0dc3d9b64f98 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:15.917637 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235899ee-6cc2-4854-af82-0dc3d9b64f98 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:15.922990 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 484/970] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:39:15.924416 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd023a-5dd2-405d-a806-d6c278e6ceb5 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:15.924705 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd023a-5dd2-405d-a806-d6c278e6ceb5 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:15.925290 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd023a-5dd2-405d-a806-d6c278e6ceb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:15.934410 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d338dbc-277a-4bab-8159-4e08526c841f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:15.935056 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d338dbc-277a-4bab-8159-4e08526c841f None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:15.935340 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d338dbc-277a-4bab-8159-4e08526c841f None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:15.935617 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d338dbc-277a-4bab-8159-4e08526c841f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:15.936283 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d338dbc-277a-4bab-8159-4e08526c841f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:15.937875 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d338dbc-277a-4bab-8159-4e08526c841f None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:15.944173 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-c5cd023a-5dd2-405d-a806-d6c278e6ceb5 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:15.945357 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-c5cd023a-5dd2-405d-a806-d6c278e6ceb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:15.947351 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-1d338dbc-277a-4bab-8159-4e08526c841f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 12:39:15.947351 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:15.947351 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 12:39:15.947351 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 12:39:15.947351 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 12:39:15.947351 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 12:39:15.947351 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 12:39:15.947351 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:39:15.947351 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 12:39:15.947351 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:15.947351 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 12:39:15.947351 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:15.947351 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:15.947351 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:15.947351 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:15.947351 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:15.947351 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:15.948431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:15.948431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:15.948431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:15.948431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:15.948431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:15.948431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:15.948431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 12:39:15.948431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 12:39:15.948431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 12:39:15.948431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 12:39:15.948431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:15.948431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:15.948431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:15.948431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:15.948431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:15.948431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:15.949275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:15.949275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:15.949275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:15.949275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:15.949275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:15.949275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:15.949275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:15.949275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:15.949275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:15.949275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:15.949275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 12:39:15.949275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:39:15.949275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 12:39:15.949275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:15.949275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 487/971] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Jul 26 12:39:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:39:15.950080 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d338dbc-277a-4bab-8159-4e08526c841f None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:16.019686 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 485/972] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 12:39:16.019686 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd023a-5dd2-405d-a806-d6c278e6ceb5 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:16.020046 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:16.055885 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d338dbc-277a-4bab-8159-4e08526c841f None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:16.090372 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294b79ed-2d4b-4e3d-b19b-7f70d0d6568c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:16.091007 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b79ed-2d4b-4e3d-b19b-7f70d0d6568c None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:16.091290 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b79ed-2d4b-4e3d-b19b-7f70d0d6568c None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:16.091565 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b79ed-2d4b-4e3d-b19b-7f70d0d6568c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:16.101613 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294b79ed-2d4b-4e3d-b19b-7f70d0d6568c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:39:16.106487 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-294b79ed-2d4b-4e3d-b19b-7f70d0d6568c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 12:39:16.106487 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:16.106487 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:16.106487 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:16.106487 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:16.106487 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:16.106487 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:16.106487 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:16.106487 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:16.106487 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:16.106487 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:16.106487 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:16.106487 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:39:16.106487 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:39:16.106487 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:39:16.106487 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:39:16.107437 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:39:16.107437 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:39:16.107437 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:39:16.107437 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:39:16.107437 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:16.107437 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:16.107437 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:16.107437 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:16.107437 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:16.107437 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:16.107437 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:16.107437 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:16.107437 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:16.107437 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:16.107437 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:16.107437 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:16.108317 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:16.108317 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:16.108317 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:16.108317 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:16.108317 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:39:16.108317 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:39:16.108317 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:39:16.108317 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:39:16.108317 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:39:16.108317 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:39:16.108317 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 12:39:16.108317 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:16.108317 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 488/973] 10.222.0.20 () {56 vars in 1120 bytes} [Tue Jul 26 12:39:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:39:16.109196 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294b79ed-2d4b-4e3d-b19b-7f70d0d6568c None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:16.119969 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd023a-5dd2-405d-a806-d6c278e6ceb5 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:16.143888 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:16.154622 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f865d-50d4-4828-b86d-1c9c1009bb23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:16.155234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f865d-50d4-4828-b86d-1c9c1009bb23 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:16.155513 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f865d-50d4-4828-b86d-1c9c1009bb23 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:16.155846 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f865d-50d4-4828-b86d-1c9c1009bb23 None admin] PATH_INFO: `/v3/services` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:16.156517 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f865d-50d4-4828-b86d-1c9c1009bb23 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:16.156817 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f865d-50d4-4828-b86d-1c9c1009bb23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:16.158219 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f865d-50d4-4828-b86d-1c9c1009bb23 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:16.162943 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 486/974] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Jul 26 12:39:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:16.163770 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f865d-50d4-4828-b86d-1c9c1009bb23 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:16.217415 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294b79ed-2d4b-4e3d-b19b-7f70d0d6568c None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:16.247510 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:16.253668 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe00c5f-d264-49e3-b07b-9895b9b63c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:16.254276 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe00c5f-d264-49e3-b07b-9895b9b63c98 None admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:16.254561 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe00c5f-d264-49e3-b07b-9895b9b63c98 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:16.254835 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe00c5f-d264-49e3-b07b-9895b9b63c98 None admin] PATH_INFO: `/v3/services` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:16.256074 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe00c5f-d264-49e3-b07b-9895b9b63c98 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:16.257440 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe00c5f-d264-49e3-b07b-9895b9b63c98 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:16.264636 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f865d-50d4-4828-b86d-1c9c1009bb23 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:16.272243 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 489/975] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Jul 26 12:39:16 2022] POST /identity/v3/services => generated 242 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:39:16.273045 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe00c5f-d264-49e3-b07b-9895b9b63c98 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:16.324701 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422efd7a-c483-4f02-8aab-2465a7f22053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:16.325342 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422efd7a-c483-4f02-8aab-2465a7f22053 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:16.325633 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422efd7a-c483-4f02-8aab-2465a7f22053 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:16.325893 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422efd7a-c483-4f02-8aab-2465a7f22053 None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:16.326979 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422efd7a-c483-4f02-8aab-2465a7f22053 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:16.326979 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422efd7a-c483-4f02-8aab-2465a7f22053 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:16.328380 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422efd7a-c483-4f02-8aab-2465a7f22053 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:16.338675 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 487/976] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Jul 26 12:39:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:16.344409 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422efd7a-c483-4f02-8aab-2465a7f22053 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:16.396472 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:16.411008 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe00c5f-d264-49e3-b07b-9895b9b63c98 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:16.452669 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102755e5-7831-480f-a51d-e534d37d2ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:16.453295 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102755e5-7831-480f-a51d-e534d37d2ba8 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:16.453578 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102755e5-7831-480f-a51d-e534d37d2ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:16.453854 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102755e5-7831-480f-a51d-e534d37d2ba8 None admin] PATH_INFO: `/v3/services` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:16.454542 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102755e5-7831-480f-a51d-e534d37d2ba8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:16.454852 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102755e5-7831-480f-a51d-e534d37d2ba8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:16.456452 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102755e5-7831-480f-a51d-e534d37d2ba8 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:16.462248 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 490/977] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Jul 26 12:39:16 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:16.467178 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422efd7a-c483-4f02-8aab-2465a7f22053 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:16.468764 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102755e5-7831-480f-a51d-e534d37d2ba8 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:16.505840 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a27f0-1f77-459d-a870-bb51a3b05cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:16.506495 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a27f0-1f77-459d-a870-bb51a3b05cfa None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:16.506774 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a27f0-1f77-459d-a870-bb51a3b05cfa None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:16.507046 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a27f0-1f77-459d-a870-bb51a3b05cfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:16.520703 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7a27f0-1f77-459d-a870-bb51a3b05cfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:16.522233 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7a27f0-1f77-459d-a870-bb51a3b05cfa None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:16.529775 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:16.538621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-aa7a27f0-1f77-459d-a870-bb51a3b05cfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:39:16.538621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:16.538621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:16.538621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:16.538621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:16.538621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:16.538621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:16.538621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:16.538621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:16.538621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:16.538621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:16.538621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:16.538621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:39:16.538621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:39:16.538621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:39:16.539554 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:39:16.539554 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:16.539554 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:16.539554 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:16.539554 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:16.539554 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:16.539554 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:16.539554 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:16.539554 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:16.539554 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:16.539554 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:16.539554 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:16.539554 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:16.539554 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:16.539554 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:16.539554 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:16.539554 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:16.540527 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:39:16.540527 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:39:16.540527 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:39:16.540527 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:16.543942 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 488/978] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:39:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:39:16.545571 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a27f0-1f77-459d-a870-bb51a3b05cfa None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:16.581355 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102755e5-7831-480f-a51d-e534d37d2ba8 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:16.620741 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41471dd-d39e-4caa-a7bd-24863a714677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:16.621363 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41471dd-d39e-4caa-a7bd-24863a714677 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:16.621648 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41471dd-d39e-4caa-a7bd-24863a714677 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:16.621933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41471dd-d39e-4caa-a7bd-24863a714677 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:16.622971 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41471dd-d39e-4caa-a7bd-24863a714677 None admin] RBAC: Adding query filter params (interface=admin, service_id=815fa546088d4a9b92b140a4ee327568, region_id=RegionOne) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:16.623289 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41471dd-d39e-4caa-a7bd-24863a714677 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:16.625574 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41471dd-d39e-4caa-a7bd-24863a714677 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:16.636675 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 491/979] 10.222.0.20 () {56 vars in 1249 bytes} [Tue Jul 26 12:39:16 2022] GET /identity/v3/endpoints?service_id=815fa546088d4a9b92b140a4ee327568&interface=admin®ion_id=RegionOne => generated 193 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:39:16.638026 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41471dd-d39e-4caa-a7bd-24863a714677 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:16.658148 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:16.661698 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a27f0-1f77-459d-a870-bb51a3b05cfa None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:16.697604 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f57e2d-fb08-4f75-b7d2-72b7120d6800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:16.698223 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f57e2d-fb08-4f75-b7d2-72b7120d6800 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:16.698503 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f57e2d-fb08-4f75-b7d2-72b7120d6800 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:16.698777 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f57e2d-fb08-4f75-b7d2-72b7120d6800 None admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:16.699465 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f57e2d-fb08-4f75-b7d2-72b7120d6800 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:16.699800 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f57e2d-fb08-4f75-b7d2-72b7120d6800 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:16.701164 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f57e2d-fb08-4f75-b7d2-72b7120d6800 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:16.718588 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 489/980] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:39:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:39:16.719877 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f57e2d-fb08-4f75-b7d2-72b7120d6800 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:16.751740 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41471dd-d39e-4caa-a7bd-24863a714677 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:16.770096 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:16.790852 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc968d-e139-41fe-9387-d7cd77ac0ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:16.791783 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc968d-e139-41fe-9387-d7cd77ac0ce2 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:16.792364 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc968d-e139-41fe-9387-d7cd77ac0ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:16.792664 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc968d-e139-41fe-9387-d7cd77ac0ce2 None admin] PATH_INFO: `/v3/services` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:16.793353 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc968d-e139-41fe-9387-d7cd77ac0ce2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:16.794879 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc968d-e139-41fe-9387-d7cd77ac0ce2 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:16.811693 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 492/981] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Jul 26 12:39:16 2022] GET /identity/v3/services => generated 1727 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:39:16.853840 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f57e2d-fb08-4f75-b7d2-72b7120d6800 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:16.896934 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149e76ea-2683-47b9-b178-d3a5f23b65e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:16.897587 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149e76ea-2683-47b9-b178-d3a5f23b65e3 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:16.897863 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149e76ea-2683-47b9-b178-d3a5f23b65e3 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:16.898132 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149e76ea-2683-47b9-b178-d3a5f23b65e3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:16.906587 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149e76ea-2683-47b9-b178-d3a5f23b65e3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:16.908081 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149e76ea-2683-47b9-b178-d3a5f23b65e3 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:16.923947 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-149e76ea-2683-47b9-b178-d3a5f23b65e3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 12:39:16.923947 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:16.923947 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:16.923947 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:16.923947 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:16.923947 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:16.923947 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:16.923947 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:16.923947 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:16.923947 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:16.923947 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:16.923947 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:16.923947 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 12:39:16.923947 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 12:39:16.923947 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 12:39:16.923947 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 12:39:16.924946 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:16.924946 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:16.924946 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 12:39:16.924946 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:39:16.924946 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 12:39:16.924946 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:39:16.924946 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:16.924946 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:16.924946 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:16.924946 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:16.924946 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:16.924946 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:16.924946 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:16.924946 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:16.924946 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:16.924946 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:16.924946 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:16.924946 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:16.931206 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:16.931206 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:16.931206 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 12:39:16.931206 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 12:39:16.931206 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 12:39:16.931206 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 12:39:16.931206 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 12:39:16.931206 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 12:39:16.931206 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 12:39:16.931206 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:16.931206 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 490/982] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Jul 26 12:39:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:39:16.933329 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc968d-e139-41fe-9387-d7cd77ac0ce2 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:17.043754 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc968d-e139-41fe-9387-d7cd77ac0ce2 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:17.077211 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc80a3-5d79-44e7-a707-0b00eaa234e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:17.077827 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc80a3-5d79-44e7-a707-0b00eaa234e9 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:17.078103 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc80a3-5d79-44e7-a707-0b00eaa234e9 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:17.078371 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc80a3-5d79-44e7-a707-0b00eaa234e9 None admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:17.079184 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecc80a3-5d79-44e7-a707-0b00eaa234e9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:17.079489 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecc80a3-5d79-44e7-a707-0b00eaa234e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:17.080946 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecc80a3-5d79-44e7-a707-0b00eaa234e9 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:17.101652 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 493/983] 10.222.0.20 () {56 vars in 1149 bytes} [Tue Jul 26 12:39:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:17.107461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149e76ea-2683-47b9-b178-d3a5f23b65e3 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:17.127680 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:17.222258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149e76ea-2683-47b9-b178-d3a5f23b65e3 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:17.259027 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecee26c-9d0a-4ea1-9b2d-61def62fdc82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:17.259651 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecee26c-9d0a-4ea1-9b2d-61def62fdc82 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:17.259940 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecee26c-9d0a-4ea1-9b2d-61def62fdc82 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:17.260210 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecee26c-9d0a-4ea1-9b2d-61def62fdc82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:17.328741 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecee26c-9d0a-4ea1-9b2d-61def62fdc82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:17.330178 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecee26c-9d0a-4ea1-9b2d-61def62fdc82 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:17.348376 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:17.429509 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 491/984] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:39:17 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 26 12:39:17.434973 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc80a3-5d79-44e7-a707-0b00eaa234e9 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:17.479369 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:17.537006 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc80a3-5d79-44e7-a707-0b00eaa234e9 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:17.572137 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0111b067-42f2-4a70-b4c2-30e168746a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:17.572771 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0111b067-42f2-4a70-b4c2-30e168746a4b None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:17.573049 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0111b067-42f2-4a70-b4c2-30e168746a4b None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:17.573314 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0111b067-42f2-4a70-b4c2-30e168746a4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:17.583968 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0111b067-42f2-4a70-b4c2-30e168746a4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:17.585782 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0111b067-42f2-4a70-b4c2-30e168746a4b None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:17.601861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-0111b067-42f2-4a70-b4c2-30e168746a4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:39:17.601861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:17.601861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:17.601861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:17.601861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:17.601861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:17.601861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:17.601861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:17.601861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:17.601861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:17.601861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:17.601861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:17.601861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:39:17.601861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:39:17.601861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:39:17.602803 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:39:17.602803 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:17.602803 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:17.602803 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:17.602803 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:17.602803 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:17.602803 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:17.602803 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:17.602803 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:17.602803 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:17.602803 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:17.602803 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:17.602803 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:17.602803 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:17.602803 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:17.602803 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:17.602803 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:17.603736 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:39:17.603736 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:39:17.603736 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:39:17.603736 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:17.604523 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 494/985] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:39:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:39:17.609207 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecee26c-9d0a-4ea1-9b2d-61def62fdc82 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:17.709382 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecee26c-9d0a-4ea1-9b2d-61def62fdc82 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:17.726146 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:17.743824 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7216ea22-7c01-4b46-92a2-88fb00eafbeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:17.744463 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216ea22-7c01-4b46-92a2-88fb00eafbeb None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:17.744745 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216ea22-7c01-4b46-92a2-88fb00eafbeb None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:17.745016 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216ea22-7c01-4b46-92a2-88fb00eafbeb None admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:17.745701 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7216ea22-7c01-4b46-92a2-88fb00eafbeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:17.746002 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7216ea22-7c01-4b46-92a2-88fb00eafbeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:17.747394 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7216ea22-7c01-4b46-92a2-88fb00eafbeb None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:17.763915 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 492/986] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:39:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:17.765428 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0111b067-42f2-4a70-b4c2-30e168746a4b None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:17.859336 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:17.869443 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0111b067-42f2-4a70-b4c2-30e168746a4b None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:17.909806 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f3c33-c2fa-4656-b69c-d29f6255c929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:17.910447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f3c33-c2fa-4656-b69c-d29f6255c929 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:17.910727 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f3c33-c2fa-4656-b69c-d29f6255c929 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:17.910999 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f3c33-c2fa-4656-b69c-d29f6255c929 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:17.918190 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f3c33-c2fa-4656-b69c-d29f6255c929 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:17.919760 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f3c33-c2fa-4656-b69c-d29f6255c929 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:17.931268 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-c11f3c33-c2fa-4656-b69c-d29f6255c929 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:39:17.931268 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:17.931268 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:17.931268 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:17.931268 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:17.931268 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:17.931268 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:17.931268 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:17.931268 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:17.931268 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:17.931268 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:17.931268 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:17.931268 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 12:39:17.931268 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 12:39:17.931268 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 12:39:17.932466 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 12:39:17.932466 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:17.932466 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:17.932466 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:17.932466 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:17.932466 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:17.932466 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:17.932466 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:17.932466 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:17.932466 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:17.932466 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:17.932466 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:17.932466 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:17.932466 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:17.932466 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:17.932466 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:17.932466 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:17.933399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 12:39:17.933399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 12:39:17.933399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 12:39:17.933399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 12:39:17.933399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 12:39:17.933399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 12:39:17.933399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:39:17.933399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:17.933399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 495/987] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:39:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:39:17.940055 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7216ea22-7c01-4b46-92a2-88fb00eafbeb None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:18.047492 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7216ea22-7c01-4b46-92a2-88fb00eafbeb None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:18.081303 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb424ac4-31a1-4dc4-b937-ed23aa962bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:18.081919 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb424ac4-31a1-4dc4-b937-ed23aa962bc5 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:18.082201 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb424ac4-31a1-4dc4-b937-ed23aa962bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:18.082475 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb424ac4-31a1-4dc4-b937-ed23aa962bc5 None admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:18.083907 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb424ac4-31a1-4dc4-b937-ed23aa962bc5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:18.084119 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb424ac4-31a1-4dc4-b937-ed23aa962bc5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:18.085517 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb424ac4-31a1-4dc4-b937-ed23aa962bc5 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:18.097625 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 493/988] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 26 12:39:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:39:18.103405 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f3c33-c2fa-4656-b69c-d29f6255c929 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:18.212781 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f3c33-c2fa-4656-b69c-d29f6255c929 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:18.246714 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60b7583-34e7-41a3-86c3-a99cacd3415e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:18.247326 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b7583-34e7-41a3-86c3-a99cacd3415e None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:18.247611 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b7583-34e7-41a3-86c3-a99cacd3415e None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:18.247915 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b7583-34e7-41a3-86c3-a99cacd3415e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:18.248700 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60b7583-34e7-41a3-86c3-a99cacd3415e None admin] RBAC: Adding query filter params (role.id=3655f7c95a7643d296b2c6a9cca87b4b, scope.project.id=644278e7e8934b37a1e7ab20d35764cd, user.id=30fc532f464843a3ad35e4d3c387bb3a) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:18.249015 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60b7583-34e7-41a3-86c3-a99cacd3415e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:18.250468 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60b7583-34e7-41a3-86c3-a99cacd3415e None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:18.257093 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 496/989] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:39:18 2022] GET /identity/v3/role_assignments?user.id=30fc532f464843a3ad35e4d3c387bb3a&scope.project.id=644278e7e8934b37a1e7ab20d35764cd&role.id=3655f7c95a7643d296b2c6a9cca87b4b => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:18.354066 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d3ec2-1daa-432b-b86a-3078d15e7556 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:18.354371 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d3ec2-1daa-432b-b86a-3078d15e7556 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:18.354558 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d3ec2-1daa-432b-b86a-3078d15e7556 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:18.356129 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 494/990] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:39:18.361655 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d2c51-882b-45ed-9926-afe8033dd65a None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:18.361949 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d2c51-882b-45ed-9926-afe8033dd65a None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:18.362228 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d2c51-882b-45ed-9926-afe8033dd65a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:18.382463 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-f92d2c51-882b-45ed-9926-afe8033dd65a None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:18.383715 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-f92d2c51-882b-45ed-9926-afe8033dd65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:18.396771 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:18.459359 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 497/991] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:18 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:39:18.481650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258d3ec2-1daa-432b-b86a-3078d15e7556 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:18.525617 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:18.591172 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258d3ec2-1daa-432b-b86a-3078d15e7556 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:18.625816 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca4ac11-ebd3-4fc1-ba44-b7994f7a028e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:18.626362 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca4ac11-ebd3-4fc1-ba44-b7994f7a028e None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:18.626639 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca4ac11-ebd3-4fc1-ba44-b7994f7a028e None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:18.626914 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca4ac11-ebd3-4fc1-ba44-b7994f7a028e None admin] PATH_INFO: `/v3/services/compute` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:18.627547 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca4ac11-ebd3-4fc1-ba44-b7994f7a028e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:18.628945 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca4ac11-ebd3-4fc1-ba44-b7994f7a028e None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:18.638697 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-5ca4ac11-ebd3-4fc1-ba44-b7994f7a028e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 12:39:18.638697 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:18.638697 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 12:39:18.638697 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 12:39:18.638697 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 12:39:18.638697 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 12:39:18.638697 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 12:39:18.638697 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:39:18.638697 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 12:39:18.638697 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:18.638697 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 12:39:18.638697 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:18.638697 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:18.638697 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:18.638697 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:18.638697 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:18.638697 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:18.639773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:18.639773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:18.639773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:18.639773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:18.639773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:18.639773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:18.639773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 12:39:18.639773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 12:39:18.639773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 12:39:18.639773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 12:39:18.639773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:18.639773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:18.639773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:18.639773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:18.639773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:18.639773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:18.641016 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:18.641016 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:18.641016 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:18.641016 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:18.641016 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:18.641016 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:18.641016 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:18.641016 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:18.641016 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:18.641016 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:18.641016 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 12:39:18.641016 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:39:18.641016 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 12:39:18.641016 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:18.643716 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 495/992] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:39:18 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:39:18.646924 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d2c51-882b-45ed-9926-afe8033dd65a None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:18.747503 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d2c51-882b-45ed-9926-afe8033dd65a None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:18.783329 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777b4737-9003-4f40-9e32-c70bea6433d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:18.784001 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b4737-9003-4f40-9e32-c70bea6433d2 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:18.784377 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b4737-9003-4f40-9e32-c70bea6433d2 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:18.784544 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b4737-9003-4f40-9e32-c70bea6433d2 None admin] PATH_INFO: `/v3/services` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:18.785232 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777b4737-9003-4f40-9e32-c70bea6433d2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:18.785509 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777b4737-9003-4f40-9e32-c70bea6433d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:18.787447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777b4737-9003-4f40-9e32-c70bea6433d2 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:18.796011 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 498/993] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Jul 26 12:39:18 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:18.798161 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca4ac11-ebd3-4fc1-ba44-b7994f7a028e None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:18.872465 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8f9da-7d0f-4155-ba1f-d1c7c2c909b2 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:18.872789 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8f9da-7d0f-4155-ba1f-d1c7c2c909b2 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:18.873020 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8f9da-7d0f-4155-ba1f-d1c7c2c909b2 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:18.874488 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 499/994] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:39:18.879221 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122100a-8c41-43fa-9d1a-8f8870be7dfd None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:18.879432 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122100a-8c41-43fa-9d1a-8f8870be7dfd None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:18.879735 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122100a-8c41-43fa-9d1a-8f8870be7dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:18.898345 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-1122100a-8c41-43fa-9d1a-8f8870be7dfd None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:18.899508 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-1122100a-8c41-43fa-9d1a-8f8870be7dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:18.902511 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:18.909384 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca4ac11-ebd3-4fc1-ba44-b7994f7a028e None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:18.911150 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:18.948878 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75622d18-0214-4b1f-b0f2-40a648e9e85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:18.949581 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75622d18-0214-4b1f-b0f2-40a648e9e85b None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:18.949769 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75622d18-0214-4b1f-b0f2-40a648e9e85b None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:18.950038 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75622d18-0214-4b1f-b0f2-40a648e9e85b None admin] PATH_INFO: `/v3/services` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:18.950724 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75622d18-0214-4b1f-b0f2-40a648e9e85b None admin] RBAC: Adding query filter params (type=compute) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:18.951031 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75622d18-0214-4b1f-b0f2-40a648e9e85b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:18.952430 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75622d18-0214-4b1f-b0f2-40a648e9e85b None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:18.958006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 496/995] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Jul 26 12:39:18 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:18.964557 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75622d18-0214-4b1f-b0f2-40a648e9e85b None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:18.978289 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 500/996] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:18 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 12:39:19.000548 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1122100a-8c41-43fa-9d1a-8f8870be7dfd None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:19.072322 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75622d18-0214-4b1f-b0f2-40a648e9e85b None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:19.106597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1122100a-8c41-43fa-9d1a-8f8870be7dfd None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:19.110114 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4172b743-36f6-4154-bf93-29c2a96eb208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:19.110931 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4172b743-36f6-4154-bf93-29c2a96eb208 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:19.111384 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4172b743-36f6-4154-bf93-29c2a96eb208 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:19.111855 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4172b743-36f6-4154-bf93-29c2a96eb208 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:19.112792 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4172b743-36f6-4154-bf93-29c2a96eb208 None admin] RBAC: Adding query filter params (interface=public, service_id=73f90b540a8b426f96a5616cb73ce8c5, region_id=RegionOne) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:19.113270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4172b743-36f6-4154-bf93-29c2a96eb208 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:19.115017 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4172b743-36f6-4154-bf93-29c2a96eb208 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:19.121061 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 497/997] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Jul 26 12:39:18 2022] GET /identity/v3/endpoints?service_id=73f90b540a8b426f96a5616cb73ce8c5&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:39:19.126413 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4172b743-36f6-4154-bf93-29c2a96eb208 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:19.143188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9a3a3-8a4c-453a-a621-a43218b7d51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:19.143684 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9a3a3-8a4c-453a-a621-a43218b7d51b None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:19.143969 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9a3a3-8a4c-453a-a621-a43218b7d51b None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:19.144240 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9a3a3-8a4c-453a-a621-a43218b7d51b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:19.144865 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e9a3a3-8a4c-453a-a621-a43218b7d51b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:19.146780 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e9a3a3-8a4c-453a-a621-a43218b7d51b None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:19.156516 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-45e9a3a3-8a4c-453a-a621-a43218b7d51b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 12:39:19.156516 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:19.156516 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 12:39:19.156516 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 12:39:19.156516 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 12:39:19.156516 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 12:39:19.156516 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 12:39:19.156516 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:39:19.156516 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 12:39:19.156516 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:19.156516 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 12:39:19.156516 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:19.156516 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:19.156516 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:19.156516 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:19.156516 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:19.156516 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:19.157642 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:19.157642 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:19.157642 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:19.157642 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:19.157642 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:19.157642 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:19.157642 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 12:39:19.157642 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 12:39:19.157642 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 12:39:19.157642 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 12:39:19.157642 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:19.157642 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:19.157642 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:19.157642 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:19.157642 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:19.157642 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:19.158486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:19.158486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:19.158486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:19.158486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:19.158486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:19.158486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:19.158486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:19.158486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:19.158486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:19.158486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:19.158486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 12:39:19.158486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:39:19.158486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 12:39:19.158486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:19.158486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 501/998] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Jul 26 12:39:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:39:19.164548 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9a3a3-8a4c-453a-a621-a43218b7d51b None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:19.169120 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:19.169618 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:19.244503 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4172b743-36f6-4154-bf93-29c2a96eb208 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:19.281681 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9a3a3-8a4c-453a-a621-a43218b7d51b None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:19.284625 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0defbce1-392d-4d3c-b7db-ae73bea34419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:19.284831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0defbce1-392d-4d3c-b7db-ae73bea34419 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:19.285116 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0defbce1-392d-4d3c-b7db-ae73bea34419 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:19.285387 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0defbce1-392d-4d3c-b7db-ae73bea34419 None admin] PATH_INFO: `/v3/services` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:19.286036 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0defbce1-392d-4d3c-b7db-ae73bea34419 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:19.287978 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0defbce1-392d-4d3c-b7db-ae73bea34419 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:19.294556 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 498/999] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Jul 26 12:39:19 2022] GET /identity/v3/services => generated 1727 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 12:39:19.320547 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6ca40-292b-430d-a353-46db36463d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:19.321220 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6ca40-292b-430d-a353-46db36463d8b None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:19.321475 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6ca40-292b-430d-a353-46db36463d8b None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:19.321749 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6ca40-292b-430d-a353-46db36463d8b None admin] PATH_INFO: `/v3/services` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:19.322447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f6ca40-292b-430d-a353-46db36463d8b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:19.322752 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f6ca40-292b-430d-a353-46db36463d8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:19.324203 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f6ca40-292b-430d-a353-46db36463d8b None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:19.330018 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 502/1000] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Jul 26 12:39:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:19.336293 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0defbce1-392d-4d3c-b7db-ae73bea34419 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:19.444286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:19.453536 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0defbce1-392d-4d3c-b7db-ae73bea34419 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:19.496891 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da7f53-9b0b-4945-abc8-5187d58f9320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:19.497511 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da7f53-9b0b-4945-abc8-5187d58f9320 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:19.497790 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da7f53-9b0b-4945-abc8-5187d58f9320 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:19.498060 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da7f53-9b0b-4945-abc8-5187d58f9320 None admin] PATH_INFO: `/v3/services` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:19.498735 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13da7f53-9b0b-4945-abc8-5187d58f9320 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:19.499038 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13da7f53-9b0b-4945-abc8-5187d58f9320 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:19.500420 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13da7f53-9b0b-4945-abc8-5187d58f9320 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:19.506194 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 499/1001] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Jul 26 12:39:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:19.513190 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6ca40-292b-430d-a353-46db36463d8b None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:19.583044 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:19.627043 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6ca40-292b-430d-a353-46db36463d8b None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:19.661122 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b82243-a5e9-434d-a205-ceecb8375df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:19.662281 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b82243-a5e9-434d-a205-ceecb8375df4 None admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:19.662556 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b82243-a5e9-434d-a205-ceecb8375df4 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:19.662822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b82243-a5e9-434d-a205-ceecb8375df4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:19.663540 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b82243-a5e9-434d-a205-ceecb8375df4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:19.664943 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b82243-a5e9-434d-a205-ceecb8375df4 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:19.687668 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 503/1002] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:39:19 2022] POST /identity/v3/endpoints => generated 325 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:39:20.336862 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90e079-461d-41b0-bdd2-4f2bdf72622b None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:20.337377 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90e079-461d-41b0-bdd2-4f2bdf72622b None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:20.337455 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90e079-461d-41b0-bdd2-4f2bdf72622b None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:20.338931 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 500/1003] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:39:20.343938 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f6e1c8-f81c-4eab-9798-2ae79d3e73fd None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:20.344326 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f6e1c8-f81c-4eab-9798-2ae79d3e73fd None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:20.344490 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f6e1c8-f81c-4eab-9798-2ae79d3e73fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:20.364266 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-62f6e1c8-f81c-4eab-9798-2ae79d3e73fd None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:20.365786 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-62f6e1c8-f81c-4eab-9798-2ae79d3e73fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:20.448341 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 504/1004] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:20 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:39:20.469988 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c90e079-461d-41b0-bdd2-4f2bdf72622b None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:20.573218 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c90e079-461d-41b0-bdd2-4f2bdf72622b None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:20.606887 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bf8a27-ee99-467b-b763-9c1e5f4d57c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:20.607525 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf8a27-ee99-467b-b763-9c1e5f4d57c5 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:20.607837 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf8a27-ee99-467b-b763-9c1e5f4d57c5 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:20.608113 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf8a27-ee99-467b-b763-9c1e5f4d57c5 None admin] PATH_INFO: `/v3/services/image` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:20.608737 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bf8a27-ee99-467b-b763-9c1e5f4d57c5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:20.610146 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bf8a27-ee99-467b-b763-9c1e5f4d57c5 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:20.619358 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-04bf8a27-ee99-467b-b763-9c1e5f4d57c5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 12:39:20.619358 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:20.619358 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 12:39:20.619358 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 12:39:20.619358 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 12:39:20.619358 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 12:39:20.619358 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 12:39:20.619358 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:39:20.619358 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 12:39:20.619358 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:20.619358 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 12:39:20.619358 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:20.619358 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:20.619358 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:20.619358 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:20.619358 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:20.619358 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:20.620347 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:20.620347 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:20.620347 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:20.620347 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:20.620347 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:20.620347 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:20.620347 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 12:39:20.620347 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 12:39:20.620347 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 12:39:20.620347 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 12:39:20.620347 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:20.620347 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:20.620347 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:20.620347 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:20.620347 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:20.620347 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:20.621201 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:20.621201 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:20.621201 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:20.621201 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:20.621201 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:20.621201 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:20.621201 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:20.621201 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:20.621201 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:20.621201 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:20.621201 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 12:39:20.621201 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:39:20.621201 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 12:39:20.621201 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:20.623796 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 501/1005] 10.222.0.20 () {56 vars in 1105 bytes} [Tue Jul 26 12:39:20 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:39:20.626457 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f6e1c8-f81c-4eab-9798-2ae79d3e73fd None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:20.682335 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:20.730875 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f6e1c8-f81c-4eab-9798-2ae79d3e73fd None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:20.766461 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9b8dc5-125b-4cab-9af1-d6bfdba7b1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:20.767089 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b8dc5-125b-4cab-9af1-d6bfdba7b1be None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:20.767370 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b8dc5-125b-4cab-9af1-d6bfdba7b1be None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:20.767663 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b8dc5-125b-4cab-9af1-d6bfdba7b1be None admin] PATH_INFO: `/v3/services` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:20.768367 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9b8dc5-125b-4cab-9af1-d6bfdba7b1be None admin] RBAC: Adding query filter params (name=image) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:20.768675 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9b8dc5-125b-4cab-9af1-d6bfdba7b1be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:20.770048 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9b8dc5-125b-4cab-9af1-d6bfdba7b1be None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:20.780286 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 505/1006] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:39:20 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:20.781299 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bf8a27-ee99-467b-b763-9c1e5f4d57c5 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:20.810656 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:20.891755 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bf8a27-ee99-467b-b763-9c1e5f4d57c5 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:20.928605 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95059888-32cf-4170-b281-7d1654a0851f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:20.929226 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95059888-32cf-4170-b281-7d1654a0851f None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:20.929508 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95059888-32cf-4170-b281-7d1654a0851f None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:20.929782 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95059888-32cf-4170-b281-7d1654a0851f None admin] PATH_INFO: `/v3/services` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:20.930460 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95059888-32cf-4170-b281-7d1654a0851f None admin] RBAC: Adding query filter params (type=image) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:20.930766 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95059888-32cf-4170-b281-7d1654a0851f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:20.932173 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95059888-32cf-4170-b281-7d1654a0851f None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:20.943700 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 502/1007] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:39:20 2022] GET /identity/v3/services?type=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:21.327689 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c14af0-94b9-4893-969c-ef84150d6f5f None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:21.328054 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c14af0-94b9-4893-969c-ef84150d6f5f None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:21.328255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c14af0-94b9-4893-969c-ef84150d6f5f None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:21.329756 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 506/1008] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:39:21.335419 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ffb49-12ab-4d98-9f21-c6570a6fa578 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:21.335734 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ffb49-12ab-4d98-9f21-c6570a6fa578 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:21.336040 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ffb49-12ab-4d98-9f21-c6570a6fa578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:21.354348 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-e15ffb49-12ab-4d98-9f21-c6570a6fa578 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:21.355536 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-e15ffb49-12ab-4d98-9f21-c6570a6fa578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:21.428038 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 503/1009] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:21 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:39:21.455672 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c14af0-94b9-4893-969c-ef84150d6f5f None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:21.561254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c14af0-94b9-4893-969c-ef84150d6f5f None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:21.583253 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:21.606253 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442db67a-cb69-4301-b873-1fef9adb6ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:21.606894 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442db67a-cb69-4301-b873-1fef9adb6ef5 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:21.607174 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442db67a-cb69-4301-b873-1fef9adb6ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:21.607445 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442db67a-cb69-4301-b873-1fef9adb6ef5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:21.608101 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442db67a-cb69-4301-b873-1fef9adb6ef5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:21.609468 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442db67a-cb69-4301-b873-1fef9adb6ef5 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:21.618686 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-442db67a-cb69-4301-b873-1fef9adb6ef5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 12:39:21.618686 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:21.618686 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 12:39:21.618686 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 12:39:21.618686 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 12:39:21.618686 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 12:39:21.618686 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 12:39:21.618686 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:39:21.618686 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 12:39:21.618686 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:21.618686 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 12:39:21.618686 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:21.618686 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:21.618686 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:21.618686 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:21.618686 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:21.618686 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:21.620010 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:21.620010 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:21.620010 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:21.620010 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:21.620010 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:21.620010 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:21.620010 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 12:39:21.620010 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 12:39:21.620010 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 12:39:21.620010 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 12:39:21.620010 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:21.620010 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:21.620010 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:21.620010 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:21.620010 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:21.620010 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:21.620867 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:21.620867 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:21.620867 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:21.620867 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:21.620867 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:21.620867 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:21.620867 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:21.620867 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:21.620867 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:21.620867 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:21.620867 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 12:39:21.620867 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:39:21.620867 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 12:39:21.620867 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:21.623713 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 507/1010] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:39:21 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:39:21.626147 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ffb49-12ab-4d98-9f21-c6570a6fa578 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:21.644827 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:21.713135 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645489e-fb0a-4f16-a23e-62bd6eb9e042 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:21.714721 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645489e-fb0a-4f16-a23e-62bd6eb9e042 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:21.714721 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645489e-fb0a-4f16-a23e-62bd6eb9e042 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:21.715686 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 508/1011] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:39:21.729231 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c011d-f050-49b3-a121-8b6fd74a1401 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:21.730136 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c011d-f050-49b3-a121-8b6fd74a1401 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:21.730922 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c011d-f050-49b3-a121-8b6fd74a1401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:21.732524 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ffb49-12ab-4d98-9f21-c6570a6fa578 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:21.756248 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-c89c011d-f050-49b3-a121-8b6fd74a1401 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:21.758527 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-c89c011d-f050-49b3-a121-8b6fd74a1401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:21.772958 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47d6967-2341-4041-86e6-1d7069908a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:21.773578 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d6967-2341-4041-86e6-1d7069908a42 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:21.773859 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d6967-2341-4041-86e6-1d7069908a42 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:21.774138 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d6967-2341-4041-86e6-1d7069908a42 None admin] PATH_INFO: `/v3/services` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:21.774883 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47d6967-2341-4041-86e6-1d7069908a42 None admin] RBAC: Adding query filter params (name=compute) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:21.775492 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47d6967-2341-4041-86e6-1d7069908a42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:21.776623 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47d6967-2341-4041-86e6-1d7069908a42 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:21.785872 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 504/1012] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Jul 26 12:39:21 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:21.787888 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47d6967-2341-4041-86e6-1d7069908a42 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:21.842050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 509/1013] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:21 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 12:39:21.865281 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89c011d-f050-49b3-a121-8b6fd74a1401 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:21.878489 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:21.902115 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47d6967-2341-4041-86e6-1d7069908a42 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:21.938388 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736f0229-b58d-4afd-822b-b94f7e8aff8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:21.939014 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736f0229-b58d-4afd-822b-b94f7e8aff8a None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:21.939294 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736f0229-b58d-4afd-822b-b94f7e8aff8a None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:21.939568 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736f0229-b58d-4afd-822b-b94f7e8aff8a None admin] PATH_INFO: `/v3/services` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:21.940273 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736f0229-b58d-4afd-822b-b94f7e8aff8a None admin] RBAC: Adding query filter params (type=compute) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:21.940573 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736f0229-b58d-4afd-822b-b94f7e8aff8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:21.941943 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736f0229-b58d-4afd-822b-b94f7e8aff8a None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:21.947677 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 505/1014] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Jul 26 12:39:21 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:39:21.952811 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736f0229-b58d-4afd-822b-b94f7e8aff8a None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:21.983253 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89c011d-f050-49b3-a121-8b6fd74a1401 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:21.998140 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:22.026115 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6bab9-240f-404b-a697-1b3c708a8a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:22.027524 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6bab9-240f-404b-a697-1b3c708a8a8e None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:22.027842 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6bab9-240f-404b-a697-1b3c708a8a8e None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:22.028124 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6bab9-240f-404b-a697-1b3c708a8a8e None admin] PATH_INFO: `/v3/domains/default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:22.029297 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a6bab9-240f-404b-a697-1b3c708a8a8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:22.030849 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a6bab9-240f-404b-a697-1b3c708a8a8e None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:22.037647 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 510/1015] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:39:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:39:22.039246 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6bab9-240f-404b-a697-1b3c708a8a8e None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:22.060386 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736f0229-b58d-4afd-822b-b94f7e8aff8a None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:22.097729 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f6926a-70a4-493f-ac22-e34a891a07a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:22.098496 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6926a-70a4-493f-ac22-e34a891a07a4 None admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:22.099456 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6926a-70a4-493f-ac22-e34a891a07a4 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:22.099763 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6926a-70a4-493f-ac22-e34a891a07a4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:22.100481 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f6926a-70a4-493f-ac22-e34a891a07a4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:22.101849 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f6926a-70a4-493f-ac22-e34a891a07a4 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:22.117945 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:22.123347 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 506/1016] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:39:21 2022] POST /identity/v3/endpoints => generated 334 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:39:22.163068 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6bab9-240f-404b-a697-1b3c708a8a8e None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:22.206561 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecfd7ae-e5fd-4864-9a11-e3e2aaab6162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:22.207193 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecfd7ae-e5fd-4864-9a11-e3e2aaab6162 None admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:22.207471 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecfd7ae-e5fd-4864-9a11-e3e2aaab6162 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:22.207782 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecfd7ae-e5fd-4864-9a11-e3e2aaab6162 None admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:22.208994 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecfd7ae-e5fd-4864-9a11-e3e2aaab6162 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:22.210479 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecfd7ae-e5fd-4864-9a11-e3e2aaab6162 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:22.249780 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 511/1017] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Jul 26 12:39:22 2022] POST /identity/v3/projects => generated 311 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:39:22.950551 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a6575-bf9f-4e51-9b1a-5bd565eef390 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:22.951114 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a6575-bf9f-4e51-9b1a-5bd565eef390 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:22.951114 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a6575-bf9f-4e51-9b1a-5bd565eef390 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:22.952858 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 507/1018] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:39:22.960026 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b4fc3-7624-4d8b-a2f2-08a12dba7b1d None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:22.960456 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b4fc3-7624-4d8b-a2f2-08a12dba7b1d None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:22.960662 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b4fc3-7624-4d8b-a2f2-08a12dba7b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:22.979048 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-905b4fc3-7624-4d8b-a2f2-08a12dba7b1d None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:22.980258 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-905b4fc3-7624-4d8b-a2f2-08a12dba7b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:23.004506 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:23.052250 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 512/1019] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:22 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:39:23.074726 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a6575-bf9f-4e51-9b1a-5bd565eef390 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:23.187347 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a6575-bf9f-4e51-9b1a-5bd565eef390 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:23.221093 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac5aac-81ba-43d0-add7-8af2255f0905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:23.221706 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac5aac-81ba-43d0-add7-8af2255f0905 None admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:23.221981 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac5aac-81ba-43d0-add7-8af2255f0905 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:23.222247 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac5aac-81ba-43d0-add7-8af2255f0905 None admin] PATH_INFO: `/v3/services` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:23.222937 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ac5aac-81ba-43d0-add7-8af2255f0905 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:23.224854 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ac5aac-81ba-43d0-add7-8af2255f0905 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:23.247666 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 508/1020] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Jul 26 12:39:23 2022] POST /identity/v3/services => generated 242 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:39:24.218710 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d432585-7424-43f8-9a74-2d344c045d46 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:24.219238 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d432585-7424-43f8-9a74-2d344c045d46 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:24.219238 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d432585-7424-43f8-9a74-2d344c045d46 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:24.220792 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 513/1021] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 12:39:24.225735 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbde66c-11fc-4d3b-8027-4c84b2a9e65c None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:24.226181 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbde66c-11fc-4d3b-8027-4c84b2a9e65c None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:24.226261 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbde66c-11fc-4d3b-8027-4c84b2a9e65c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:24.244768 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-fdbde66c-11fc-4d3b-8027-4c84b2a9e65c None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:24.245942 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-fdbde66c-11fc-4d3b-8027-4c84b2a9e65c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:24.304066 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:24.324064 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 509/1022] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:24 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:39:24.327960 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd95ad3-236f-4cbf-9bf9-d8de08e95bd9 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:24.328249 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd95ad3-236f-4cbf-9bf9-d8de08e95bd9 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:24.328526 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd95ad3-236f-4cbf-9bf9-d8de08e95bd9 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:24.329996 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 514/1023] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:39:24.334974 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2c63e-7d1d-491d-8f14-6af6bc17a104 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:24.335263 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2c63e-7d1d-491d-8f14-6af6bc17a104 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:24.335535 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2c63e-7d1d-491d-8f14-6af6bc17a104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:24.346436 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd95ad3-236f-4cbf-9bf9-d8de08e95bd9 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:24.354080 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-e5d2c63e-7d1d-491d-8f14-6af6bc17a104 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:24.355326 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-e5d2c63e-7d1d-491d-8f14-6af6bc17a104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:24.429717 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 510/1024] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:24 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 12:39:24.454400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d2c63e-7d1d-491d-8f14-6af6bc17a104 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:24.458237 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd95ad3-236f-4cbf-9bf9-d8de08e95bd9 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:24.495141 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06698107-7c0f-4b97-b367-a7f4a6d9ecb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:24.495693 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06698107-7c0f-4b97-b367-a7f4a6d9ecb9 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:24.495986 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06698107-7c0f-4b97-b367-a7f4a6d9ecb9 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:24.496258 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06698107-7c0f-4b97-b367-a7f4a6d9ecb9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:24.507306 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06698107-7c0f-4b97-b367-a7f4a6d9ecb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:39:24.512603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-06698107-7c0f-4b97-b367-a7f4a6d9ecb9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 12:39:24.512603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:24.512603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:24.512603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:24.512603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:24.512603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:24.512603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:24.512603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:24.512603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:24.512603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:24.512603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:24.512603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:24.512603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:39:24.512603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:39:24.512603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:39:24.512603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:39:24.513551 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:39:24.513551 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:39:24.513551 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:39:24.513551 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:39:24.513551 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:24.513551 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:24.513551 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:24.513551 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:24.513551 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:24.513551 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:24.513551 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:24.513551 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:24.513551 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:24.513551 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:24.513551 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:24.513551 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:24.514441 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:24.514441 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:24.514441 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:24.514441 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:24.514441 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:39:24.514441 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:39:24.514441 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:39:24.514441 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:39:24.514441 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:39:24.514441 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:39:24.514441 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 12:39:24.514441 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:24.514441 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 515/1025] 10.222.0.20 () {56 vars in 1120 bytes} [Tue Jul 26 12:39:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:39:24.519980 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06698107-7c0f-4b97-b367-a7f4a6d9ecb9 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:24.542834 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:24.570043 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d2c63e-7d1d-491d-8f14-6af6bc17a104 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:24.587660 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:24.609321 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd610d52-fe10-4479-a9f6-03fe73eaa5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:24.609968 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd610d52-fe10-4479-a9f6-03fe73eaa5a4 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:24.610247 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd610d52-fe10-4479-a9f6-03fe73eaa5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:24.610518 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd610d52-fe10-4479-a9f6-03fe73eaa5a4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:24.611699 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd610d52-fe10-4479-a9f6-03fe73eaa5a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:24.613146 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd610d52-fe10-4479-a9f6-03fe73eaa5a4 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:24.615808 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 511/1026] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:39:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:39:24.621209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd610d52-fe10-4479-a9f6-03fe73eaa5a4 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:24.639598 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06698107-7c0f-4b97-b367-a7f4a6d9ecb9 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:24.678084 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e19f269-8911-48fb-b0b5-b1f340d121d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:24.678799 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19f269-8911-48fb-b0b5-b1f340d121d2 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:24.679148 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19f269-8911-48fb-b0b5-b1f340d121d2 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:24.679489 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19f269-8911-48fb-b0b5-b1f340d121d2 None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:24.680320 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e19f269-8911-48fb-b0b5-b1f340d121d2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:24.680629 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e19f269-8911-48fb-b0b5-b1f340d121d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:24.682014 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e19f269-8911-48fb-b0b5-b1f340d121d2 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:24.695597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 516/1027] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Jul 26 12:39:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:39:24.697316 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e19f269-8911-48fb-b0b5-b1f340d121d2 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:24.733428 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd610d52-fe10-4479-a9f6-03fe73eaa5a4 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:24.769900 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e760a0-3b41-4d31-b027-468db062cd17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:24.770530 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e760a0-3b41-4d31-b027-468db062cd17 None admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:24.771354 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e760a0-3b41-4d31-b027-468db062cd17 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:24.771627 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e760a0-3b41-4d31-b027-468db062cd17 None admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:24.772333 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e760a0-3b41-4d31-b027-468db062cd17 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:24.773759 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e760a0-3b41-4d31-b027-468db062cd17 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:24.788829 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:24.796943 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 512/1028] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Jul 26 12:39:24 2022] POST /identity/v3/users => generated 285 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:39:24.812938 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e19f269-8911-48fb-b0b5-b1f340d121d2 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:24.854283 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef65bf-386e-4457-b46e-9fa2869a1957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:24.854956 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef65bf-386e-4457-b46e-9fa2869a1957 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:24.855238 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef65bf-386e-4457-b46e-9fa2869a1957 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:24.855510 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef65bf-386e-4457-b46e-9fa2869a1957 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:24.869724 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ef65bf-386e-4457-b46e-9fa2869a1957 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:24.871225 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ef65bf-386e-4457-b46e-9fa2869a1957 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:24.888303 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-14ef65bf-386e-4457-b46e-9fa2869a1957 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:39:24.888303 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:24.888303 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:24.888303 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:24.888303 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:24.888303 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:24.888303 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:24.888303 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:24.888303 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:24.888303 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:24.888303 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:24.888303 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:24.888303 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:39:24.888303 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:39:24.888303 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:39:24.889309 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:39:24.889309 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:24.889309 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:24.889309 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:24.889309 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:24.889309 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:24.889309 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:24.889309 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:24.889309 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:24.889309 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:24.889309 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:24.889309 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:24.889309 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:24.889309 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:24.889309 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:24.889309 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:24.889309 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:24.895315 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:39:24.895315 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:39:24.895315 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:39:24.895315 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:24.895315 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 517/1029] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:39:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:39:24.897611 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e760a0-3b41-4d31-b027-468db062cd17 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:24.945812 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:25.012026 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e760a0-3b41-4d31-b027-468db062cd17 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:25.050450 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6fa1ad-8cfc-4a16-913d-ae451f9568ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:25.051076 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fa1ad-8cfc-4a16-913d-ae451f9568ae None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:25.051358 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fa1ad-8cfc-4a16-913d-ae451f9568ae None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:25.051630 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fa1ad-8cfc-4a16-913d-ae451f9568ae None admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:25.052479 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6fa1ad-8cfc-4a16-913d-ae451f9568ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:25.052784 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6fa1ad-8cfc-4a16-913d-ae451f9568ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:25.054153 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6fa1ad-8cfc-4a16-913d-ae451f9568ae None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:25.066689 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 513/1030] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:39:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:25.072152 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef65bf-386e-4457-b46e-9fa2869a1957 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:25.184597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef65bf-386e-4457-b46e-9fa2869a1957 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:25.216994 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:25.222488 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6bc305-7d7a-4029-bf3d-54325360f6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:25.223192 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6bc305-7d7a-4029-bf3d-54325360f6f7 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:25.223407 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6bc305-7d7a-4029-bf3d-54325360f6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:25.223744 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6bc305-7d7a-4029-bf3d-54325360f6f7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:25.232659 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6bc305-7d7a-4029-bf3d-54325360f6f7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:25.234160 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6bc305-7d7a-4029-bf3d-54325360f6f7 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:25.246133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-4b6bc305-7d7a-4029-bf3d-54325360f6f7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 12:39:25.246133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:25.246133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:25.246133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:25.246133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:25.246133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:25.246133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:25.246133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:25.246133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:25.246133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:25.246133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:25.246133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:25.246133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 12:39:25.246133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 12:39:25.246133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 12:39:25.246133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 12:39:25.247177 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:25.247177 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:25.247177 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 12:39:25.247177 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:39:25.247177 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 12:39:25.247177 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:39:25.247177 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:25.247177 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:25.247177 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:25.247177 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:25.247177 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:25.247177 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:25.247177 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:25.247177 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:25.247177 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:25.247177 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:25.247177 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:25.247177 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:25.248172 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:25.248172 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:25.248172 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 12:39:25.248172 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 12:39:25.248172 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 12:39:25.248172 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 12:39:25.248172 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 12:39:25.248172 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 12:39:25.248172 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 12:39:25.248172 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:25.248172 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 518/1031] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Jul 26 12:39:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:39:25.254906 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6fa1ad-8cfc-4a16-913d-ae451f9568ae None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:25.258688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b451b6fb-6804-437a-a735-4ced914b4015 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:25.258977 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b451b6fb-6804-437a-a735-4ced914b4015 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:25.259880 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b451b6fb-6804-437a-a735-4ced914b4015 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:25.261244 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 519/1032] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 12:39:25.266886 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99911e-a95c-4570-9118-2b2ecea523c5 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:25.267118 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99911e-a95c-4570-9118-2b2ecea523c5 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:25.267400 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99911e-a95c-4570-9118-2b2ecea523c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:25.287412 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-5a99911e-a95c-4570-9118-2b2ecea523c5 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:25.288648 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-5a99911e-a95c-4570-9118-2b2ecea523c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:25.358012 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 520/1033] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:25 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 12:39:25.368521 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:25.372334 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6fa1ad-8cfc-4a16-913d-ae451f9568ae None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:25.380745 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a99911e-a95c-4570-9118-2b2ecea523c5 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:25.410035 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c699f984-7b18-442b-841b-a62e1cc6727d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:25.410649 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699f984-7b18-442b-841b-a62e1cc6727d None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:25.410925 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699f984-7b18-442b-841b-a62e1cc6727d None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:25.411711 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699f984-7b18-442b-841b-a62e1cc6727d None admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:25.412512 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c699f984-7b18-442b-841b-a62e1cc6727d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:25.412817 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c699f984-7b18-442b-841b-a62e1cc6727d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:25.414230 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c699f984-7b18-442b-841b-a62e1cc6727d None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:25.436909 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 514/1034] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Jul 26 12:39:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:25.438294 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c699f984-7b18-442b-841b-a62e1cc6727d None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:25.496183 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a99911e-a95c-4570-9118-2b2ecea523c5 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:25.521325 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:25.534619 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d7297-b5f4-4cb2-b1a1-e97a60dd2b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:25.535267 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d7297-b5f4-4cb2-b1a1-e97a60dd2b48 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:25.535550 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d7297-b5f4-4cb2-b1a1-e97a60dd2b48 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:25.535858 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d7297-b5f4-4cb2-b1a1-e97a60dd2b48 None admin] PATH_INFO: `/v3/services/image` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:25.536529 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d7297-b5f4-4cb2-b1a1-e97a60dd2b48 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:25.537909 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d7297-b5f4-4cb2-b1a1-e97a60dd2b48 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:25.543879 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c699f984-7b18-442b-841b-a62e1cc6727d None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:25.547494 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-4a8d7297-b5f4-4cb2-b1a1-e97a60dd2b48 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 12:39:25.547494 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:25.547494 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 12:39:25.547494 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 12:39:25.547494 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 12:39:25.547494 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 12:39:25.547494 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 12:39:25.547494 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:39:25.547494 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 12:39:25.547494 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:25.547494 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 12:39:25.547494 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:25.547494 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:25.547494 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:25.547494 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:25.547494 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:25.547494 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:25.549337 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:25.549337 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:25.549337 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:25.549337 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:25.549337 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:25.549337 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:25.549337 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 12:39:25.549337 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 12:39:25.549337 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 12:39:25.549337 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 12:39:25.549337 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:25.549337 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:25.549337 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:25.549337 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:25.549337 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:25.549337 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:25.551625 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:25.551625 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:25.551625 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:25.551625 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:25.551625 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:25.551625 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:25.551625 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:25.551625 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:25.551625 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:25.551625 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:25.551625 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 12:39:25.551625 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:39:25.551625 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 12:39:25.551625 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:25.551625 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 521/1035] 10.222.0.20 () {56 vars in 1105 bytes} [Tue Jul 26 12:39:25 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:39:25.556565 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d7297-b5f4-4cb2-b1a1-e97a60dd2b48 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:25.585297 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5945ef-768f-4d62-bc3d-0d99de0b7ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:25.585921 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5945ef-768f-4d62-bc3d-0d99de0b7ee2 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:25.586198 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5945ef-768f-4d62-bc3d-0d99de0b7ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:25.586468 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5945ef-768f-4d62-bc3d-0d99de0b7ee2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:25.615114 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:25.658157 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5945ef-768f-4d62-bc3d-0d99de0b7ee2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:25.659603 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5945ef-768f-4d62-bc3d-0d99de0b7ee2 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:25.674368 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d7297-b5f4-4cb2-b1a1-e97a60dd2b48 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:25.712043 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735fa19-5082-4d1f-b5f9-0c22edd524d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:25.712651 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735fa19-5082-4d1f-b5f9-0c22edd524d4 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:25.712931 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735fa19-5082-4d1f-b5f9-0c22edd524d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:25.713201 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735fa19-5082-4d1f-b5f9-0c22edd524d4 None admin] PATH_INFO: `/v3/services` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:25.713986 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1735fa19-5082-4d1f-b5f9-0c22edd524d4 None admin] RBAC: Adding query filter params (name=image) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:25.714298 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1735fa19-5082-4d1f-b5f9-0c22edd524d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:25.715688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1735fa19-5082-4d1f-b5f9-0c22edd524d4 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:25.726215 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 522/1036] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:39:25 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:39:25.727326 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735fa19-5082-4d1f-b5f9-0c22edd524d4 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:25.768610 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 515/1037] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:39:25 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 12:39:25.774400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5945ef-768f-4d62-bc3d-0d99de0b7ee2 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:25.784504 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:25.836745 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:25.837400 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735fa19-5082-4d1f-b5f9-0c22edd524d4 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:25.873046 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fbfbd4-1bbd-49ba-b886-caff17dff5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:25.873665 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fbfbd4-1bbd-49ba-b886-caff17dff5d2 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:25.873945 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fbfbd4-1bbd-49ba-b886-caff17dff5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:25.874215 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fbfbd4-1bbd-49ba-b886-caff17dff5d2 None admin] PATH_INFO: `/v3/services` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:25.874886 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fbfbd4-1bbd-49ba-b886-caff17dff5d2 None admin] RBAC: Adding query filter params (type=image) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:25.875193 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fbfbd4-1bbd-49ba-b886-caff17dff5d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:25.876592 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fbfbd4-1bbd-49ba-b886-caff17dff5d2 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:25.881693 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5945ef-768f-4d62-bc3d-0d99de0b7ee2 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:25.886705 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 523/1038] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:39:25 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:25.887813 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fbfbd4-1bbd-49ba-b886-caff17dff5d2 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:25.918523 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6dc025-3701-4667-8a53-c480132e529c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:25.919141 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6dc025-3701-4667-8a53-c480132e529c None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:25.919419 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6dc025-3701-4667-8a53-c480132e529c None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:25.919719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6dc025-3701-4667-8a53-c480132e529c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:25.930406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6dc025-3701-4667-8a53-c480132e529c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:25.931985 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6dc025-3701-4667-8a53-c480132e529c None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:25.952831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-4c6dc025-3701-4667-8a53-c480132e529c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:39:25.952831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:25.952831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:25.952831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:25.952831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:25.952831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:25.952831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:25.952831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:25.952831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:25.952831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:25.952831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:25.952831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:25.952831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:39:25.952831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:39:25.952831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:39:25.953789 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:39:25.953789 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:25.953789 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:25.953789 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:25.953789 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:25.953789 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:25.953789 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:25.953789 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:25.953789 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:25.953789 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:25.953789 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:25.953789 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:25.953789 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:25.953789 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:25.953789 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:25.953789 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:25.953789 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:25.954672 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:39:25.954672 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:39:25.954672 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:39:25.954672 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:25.954903 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 516/1039] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:39:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:39:25.959940 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6dc025-3701-4667-8a53-c480132e529c None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:26.001886 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fbfbd4-1bbd-49ba-b886-caff17dff5d2 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:26.017541 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:26.039714 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23490bb5-3728-4204-a5c2-8fc735313006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:26.040640 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23490bb5-3728-4204-a5c2-8fc735313006 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:26.040922 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23490bb5-3728-4204-a5c2-8fc735313006 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:26.041195 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23490bb5-3728-4204-a5c2-8fc735313006 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:26.041952 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23490bb5-3728-4204-a5c2-8fc735313006 None admin] RBAC: Adding query filter params (interface=public, service_id=a9f43c71408148a49d49f2d6e47ebb9d, region_id=RegionOne) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:26.042263 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23490bb5-3728-4204-a5c2-8fc735313006 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:26.043660 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23490bb5-3728-4204-a5c2-8fc735313006 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:26.049294 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 524/1040] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Jul 26 12:39:25 2022] GET /identity/v3/endpoints?service_id=a9f43c71408148a49d49f2d6e47ebb9d&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:26.055483 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23490bb5-3728-4204-a5c2-8fc735313006 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:26.079040 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6dc025-3701-4667-8a53-c480132e529c None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:26.092625 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:26.122939 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333ccc2a-c531-4757-98e2-83a41c15183a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:26.123840 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ccc2a-c531-4757-98e2-83a41c15183a None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:26.124124 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ccc2a-c531-4757-98e2-83a41c15183a None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:26.124393 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ccc2a-c531-4757-98e2-83a41c15183a None admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:26.125233 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333ccc2a-c531-4757-98e2-83a41c15183a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:26.126259 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333ccc2a-c531-4757-98e2-83a41c15183a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:26.127627 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333ccc2a-c531-4757-98e2-83a41c15183a None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:26.146944 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 517/1041] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:39:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:39:26.148012 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333ccc2a-c531-4757-98e2-83a41c15183a None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:26.160774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23490bb5-3728-4204-a5c2-8fc735313006 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:26.199321 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920cc64b-a7bd-43b1-8e82-7e39e679c7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:26.199974 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920cc64b-a7bd-43b1-8e82-7e39e679c7a6 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:26.200260 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920cc64b-a7bd-43b1-8e82-7e39e679c7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:26.200751 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920cc64b-a7bd-43b1-8e82-7e39e679c7a6 None admin] PATH_INFO: `/v3/services` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:26.201188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920cc64b-a7bd-43b1-8e82-7e39e679c7a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:26.202592 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920cc64b-a7bd-43b1-8e82-7e39e679c7a6 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:26.209509 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 525/1042] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Jul 26 12:39:26 2022] GET /identity/v3/services => generated 1957 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:39:26.269301 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333ccc2a-c531-4757-98e2-83a41c15183a None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:26.309215 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b372e2e-abb6-41d6-88a8-e0d7a89a193d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:26.309783 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b372e2e-abb6-41d6-88a8-e0d7a89a193d None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:26.310060 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b372e2e-abb6-41d6-88a8-e0d7a89a193d None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:26.310330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b372e2e-abb6-41d6-88a8-e0d7a89a193d None admin] PATH_INFO: `/v3/projects/service` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:26.318042 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b372e2e-abb6-41d6-88a8-e0d7a89a193d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:26.319516 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b372e2e-abb6-41d6-88a8-e0d7a89a193d None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:26.333144 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-1b372e2e-abb6-41d6-88a8-e0d7a89a193d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:39:26.333144 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:26.333144 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:26.333144 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:26.333144 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:26.333144 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:26.333144 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:26.333144 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:26.333144 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:26.333144 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:26.333144 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:26.333144 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:26.333144 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 12:39:26.333144 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 12:39:26.333144 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 12:39:26.334303 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 12:39:26.334303 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:26.334303 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:26.334303 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:26.334303 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:26.334303 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:26.334303 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:26.334303 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:26.334303 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:26.334303 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:26.334303 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:26.334303 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:26.334303 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:26.334303 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:26.334303 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:26.334303 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:26.334303 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:26.335934 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 12:39:26.335934 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 12:39:26.335934 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 12:39:26.335934 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 12:39:26.335934 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 12:39:26.335934 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 12:39:26.335934 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:39:26.335934 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:26.335934 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 518/1043] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:39:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:39:26.340482 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920cc64b-a7bd-43b1-8e82-7e39e679c7a6 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:26.390636 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:26.458750 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920cc64b-a7bd-43b1-8e82-7e39e679c7a6 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:26.493632 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fffc247-3fa8-4974-87f9-49d99a05999b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:26.494241 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffc247-3fa8-4974-87f9-49d99a05999b None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:26.494516 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffc247-3fa8-4974-87f9-49d99a05999b None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:26.494785 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffc247-3fa8-4974-87f9-49d99a05999b None admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:26.495530 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fffc247-3fa8-4974-87f9-49d99a05999b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:26.495929 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fffc247-3fa8-4974-87f9-49d99a05999b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:26.497341 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fffc247-3fa8-4974-87f9-49d99a05999b None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:26.509795 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 526/1044] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 26 12:39:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:26.515941 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b372e2e-abb6-41d6-88a8-e0d7a89a193d None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:26.547178 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:26.618446 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b372e2e-abb6-41d6-88a8-e0d7a89a193d None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:26.654037 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d58d7-c85b-4e92-9295-cd8016ec85c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:26.654651 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d58d7-c85b-4e92-9295-cd8016ec85c3 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:26.654926 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d58d7-c85b-4e92-9295-cd8016ec85c3 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:26.655193 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d58d7-c85b-4e92-9295-cd8016ec85c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:26.655999 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2d58d7-c85b-4e92-9295-cd8016ec85c3 None admin] RBAC: Adding query filter params (role.id=3655f7c95a7643d296b2c6a9cca87b4b, scope.project.id=644278e7e8934b37a1e7ab20d35764cd, user.id=0f5d6f3c12244c0faf26048513dc49e7) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:26.656301 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2d58d7-c85b-4e92-9295-cd8016ec85c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:26.657726 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2d58d7-c85b-4e92-9295-cd8016ec85c3 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:26.671684 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 519/1045] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:39:26 2022] GET /identity/v3/role_assignments?user.id=0f5d6f3c12244c0faf26048513dc49e7&scope.project.id=644278e7e8934b37a1e7ab20d35764cd&role.id=3655f7c95a7643d296b2c6a9cca87b4b => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:26.926247 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab165d3-1005-43ba-bf80-8c2abf8fa4a9 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:26.926565 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab165d3-1005-43ba-bf80-8c2abf8fa4a9 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:26.926933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab165d3-1005-43ba-bf80-8c2abf8fa4a9 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:26.928375 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 527/1046] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:39:26.933592 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb1641-77e4-4380-97d3-32bfe82cc178 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:26.933911 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb1641-77e4-4380-97d3-32bfe82cc178 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:26.934190 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb1641-77e4-4380-97d3-32bfe82cc178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:26.953206 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-22eb1641-77e4-4380-97d3-32bfe82cc178 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:26.954405 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-22eb1641-77e4-4380-97d3-32bfe82cc178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:27.027692 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 520/1047] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:26 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:39:27.051135 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab165d3-1005-43ba-bf80-8c2abf8fa4a9 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:27.158244 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab165d3-1005-43ba-bf80-8c2abf8fa4a9 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:27.159822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:27.201651 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da09ac-381c-479b-999e-d34762fc8f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:27.202284 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da09ac-381c-479b-999e-d34762fc8f72 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:27.202566 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da09ac-381c-479b-999e-d34762fc8f72 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:27.202840 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da09ac-381c-479b-999e-d34762fc8f72 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:27.213845 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88da09ac-381c-479b-999e-d34762fc8f72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:39:27.218774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-88da09ac-381c-479b-999e-d34762fc8f72 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:39:27.218774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:27.218774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:27.218774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:27.218774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:27.218774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:27.218774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:27.218774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:27.218774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:27.218774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:27.218774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:27.218774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:27.218774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:39:27.218774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:39:27.218774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:39:27.218774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:39:27.219953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:39:27.219953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:39:27.219953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:39:27.219953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:39:27.219953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:27.219953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:27.219953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:27.219953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:27.219953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:27.219953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:27.219953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:27.219953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:27.219953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:27.219953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:27.219953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:27.219953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:27.225918 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:27.225918 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:27.225918 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:27.225918 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:27.225918 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:39:27.225918 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:39:27.225918 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:39:27.225918 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:39:27.225918 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:39:27.225918 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:39:27.225918 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:39:27.225918 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:27.225918 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 528/1048] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 26 12:39:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:39:27.228045 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb1641-77e4-4380-97d3-32bfe82cc178 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:27.254543 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:27.333809 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb1641-77e4-4380-97d3-32bfe82cc178 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:27.371217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed61aa4-5877-4229-9474-ccbf4599fc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:27.372217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed61aa4-5877-4229-9474-ccbf4599fc6c None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:27.372724 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed61aa4-5877-4229-9474-ccbf4599fc6c None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:27.373215 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed61aa4-5877-4229-9474-ccbf4599fc6c None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:27.374136 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed61aa4-5877-4229-9474-ccbf4599fc6c None admin] RBAC: Adding query filter params (name=admin) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:27.374643 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed61aa4-5877-4229-9474-ccbf4599fc6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:27.376256 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed61aa4-5877-4229-9474-ccbf4599fc6c None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:27.385908 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 521/1049] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:39:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:27.391708 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da09ac-381c-479b-999e-d34762fc8f72 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:27.499713 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da09ac-381c-479b-999e-d34762fc8f72 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:27.532911 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a97c6-6f6b-45b1-a367-de1418575c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:27.533537 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a97c6-6f6b-45b1-a367-de1418575c6b None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:27.533811 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a97c6-6f6b-45b1-a367-de1418575c6b None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:27.534078 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a97c6-6f6b-45b1-a367-de1418575c6b None admin] PATH_INFO: `/v3/users/3bcd4efbdb0d4da09af979ef3023fb7e` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:27.547517 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455a97c6-6f6b-45b1-a367-de1418575c6b None admin] RBAC: Authorizing `identity:get_user(user_id=3bcd4efbdb0d4da09af979ef3023fb7e)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:27.548984 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455a97c6-6f6b-45b1-a367-de1418575c6b None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:27.550760 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 529/1050] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:39:27 2022] GET /identity/v3/users/3bcd4efbdb0d4da09af979ef3023fb7e => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:27.556510 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed61aa4-5877-4229-9474-ccbf4599fc6c None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:27.647788 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:27.664232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed61aa4-5877-4229-9474-ccbf4599fc6c None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:27.698581 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eca14d-a8b5-4958-9214-9b57720a1b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:27.699198 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eca14d-a8b5-4958-9214-9b57720a1b54 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:27.699477 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eca14d-a8b5-4958-9214-9b57720a1b54 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:27.699772 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eca14d-a8b5-4958-9214-9b57720a1b54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:27.771353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eca14d-a8b5-4958-9214-9b57720a1b54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:27.772818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eca14d-a8b5-4958-9214-9b57720a1b54 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:27.860798 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:27.883024 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 522/1051] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:39:27 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 12:39:27.884362 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a97c6-6f6b-45b1-a367-de1418575c6b None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:27.907884 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:27.992819 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a97c6-6f6b-45b1-a367-de1418575c6b None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:28.028264 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f330f83-be84-4cc4-9b0c-a77777ec7b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:28.028963 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f330f83-be84-4cc4-9b0c-a77777ec7b57 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:28.029341 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f330f83-be84-4cc4-9b0c-a77777ec7b57 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:28.029732 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f330f83-be84-4cc4-9b0c-a77777ec7b57 None admin] PATH_INFO: `/v3/projects/149fa0d3b3664a1ebcd7741ec11f0256` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:28.030975 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f330f83-be84-4cc4-9b0c-a77777ec7b57 None admin] RBAC: Authorizing `identity:get_project(project_id=149fa0d3b3664a1ebcd7741ec11f0256)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:28.032531 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f330f83-be84-4cc4-9b0c-a77777ec7b57 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:28.034439 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 530/1052] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:39:27 2022] GET /identity/v3/projects/149fa0d3b3664a1ebcd7741ec11f0256 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:28.040352 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eca14d-a8b5-4958-9214-9b57720a1b54 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:28.143285 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eca14d-a8b5-4958-9214-9b57720a1b54 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:28.188256 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448cbe58-2764-438a-bb17-1adcaf5c18f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:28.188879 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448cbe58-2764-438a-bb17-1adcaf5c18f0 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:28.189156 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448cbe58-2764-438a-bb17-1adcaf5c18f0 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:28.189423 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448cbe58-2764-438a-bb17-1adcaf5c18f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:28.190201 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448cbe58-2764-438a-bb17-1adcaf5c18f0 None admin] RBAC: Adding query filter params (role.id=0e78444e43c744668178971af4087ca0, scope.project.id=149fa0d3b3664a1ebcd7741ec11f0256, user.id=3bcd4efbdb0d4da09af979ef3023fb7e) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:28.190507 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448cbe58-2764-438a-bb17-1adcaf5c18f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:28.191951 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448cbe58-2764-438a-bb17-1adcaf5c18f0 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:28.198472 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 523/1053] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:39:28 2022] GET /identity/v3/role_assignments?user.id=3bcd4efbdb0d4da09af979ef3023fb7e&scope.project.id=149fa0d3b3664a1ebcd7741ec11f0256&role.id=0e78444e43c744668178971af4087ca0 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:28.245375 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9180d1c-b898-4baf-a5ea-d4d8dad3825d None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:28.245720 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9180d1c-b898-4baf-a5ea-d4d8dad3825d None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:28.245934 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9180d1c-b898-4baf-a5ea-d4d8dad3825d None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:28.247515 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 531/1054] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:39:28.253329 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f02414-7fa8-4fe9-8aa0-56c1cd9f7431 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:28.253654 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f02414-7fa8-4fe9-8aa0-56c1cd9f7431 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:28.253878 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f02414-7fa8-4fe9-8aa0-56c1cd9f7431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:28.272911 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-d4f02414-7fa8-4fe9-8aa0-56c1cd9f7431 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:28.274120 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-d4f02414-7fa8-4fe9-8aa0-56c1cd9f7431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:28.327917 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:28.356091 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 524/1055] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:28 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 12:39:28.379370 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9180d1c-b898-4baf-a5ea-d4d8dad3825d None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:28.475021 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:28.489931 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9180d1c-b898-4baf-a5ea-d4d8dad3825d None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:28.525801 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb05ff8-0d45-4ff5-88cb-85130b82a8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:28.526438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb05ff8-0d45-4ff5-88cb-85130b82a8d3 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:28.526720 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb05ff8-0d45-4ff5-88cb-85130b82a8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:28.526995 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb05ff8-0d45-4ff5-88cb-85130b82a8d3 None admin] PATH_INFO: `/v3/services/image` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:28.527627 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb05ff8-0d45-4ff5-88cb-85130b82a8d3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:28.529016 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb05ff8-0d45-4ff5-88cb-85130b82a8d3 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:28.538573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-adb05ff8-0d45-4ff5-88cb-85130b82a8d3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 12:39:28.538573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:28.538573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 12:39:28.538573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 12:39:28.538573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 12:39:28.538573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 12:39:28.538573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 12:39:28.538573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:39:28.538573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 12:39:28.538573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:28.538573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 12:39:28.538573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:28.538573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:28.538573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:28.538573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:28.538573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:28.538573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:28.539603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:28.539603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:28.539603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:28.539603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:28.539603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:28.539603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:28.539603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 12:39:28.539603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 12:39:28.539603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 12:39:28.539603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 12:39:28.539603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:28.539603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:28.539603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:28.539603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:28.539603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:28.539603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:28.541188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:28.541188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:28.541188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:28.541188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:28.541188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:28.541188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:28.541188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:28.541188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:28.541188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:28.541188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:28.541188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 12:39:28.541188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:39:28.541188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 12:39:28.541188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:28.541188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 532/1056] 10.222.0.20 () {56 vars in 1105 bytes} [Tue Jul 26 12:39:28 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:39:28.548206 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f02414-7fa8-4fe9-8aa0-56c1cd9f7431 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:28.650312 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f02414-7fa8-4fe9-8aa0-56c1cd9f7431 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:28.684845 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219cb107-8223-4135-8915-ad2cb1bee53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:28.685464 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219cb107-8223-4135-8915-ad2cb1bee53e None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:28.685739 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219cb107-8223-4135-8915-ad2cb1bee53e None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:28.686006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219cb107-8223-4135-8915-ad2cb1bee53e None admin] PATH_INFO: `/v3/services` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:28.686672 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219cb107-8223-4135-8915-ad2cb1bee53e None admin] RBAC: Adding query filter params (name=image) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:28.686970 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219cb107-8223-4135-8915-ad2cb1bee53e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:28.688352 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219cb107-8223-4135-8915-ad2cb1bee53e None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:28.698730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 525/1057] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:39:28 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:28.699979 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb05ff8-0d45-4ff5-88cb-85130b82a8d3 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:28.712184 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86aff2-1055-4663-88f0-62c551083dd3 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:28.712804 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86aff2-1055-4663-88f0-62c551083dd3 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:28.713287 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86aff2-1055-4663-88f0-62c551083dd3 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:28.715950 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 526/1058] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:39:28.721079 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c2c7a3-f149-41c7-be98-3dc678516c6f None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:28.721365 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c2c7a3-f149-41c7-be98-3dc678516c6f None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:28.721646 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c2c7a3-f149-41c7-be98-3dc678516c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:28.744975 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-f4c2c7a3-f149-41c7-be98-3dc678516c6f None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:28.746179 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-f4c2c7a3-f149-41c7-be98-3dc678516c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:28.794008 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:28.822997 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 527/1059] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:28 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 12:39:28.825139 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb05ff8-0d45-4ff5-88cb-85130b82a8d3 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:28.845352 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c2c7a3-f149-41c7-be98-3dc678516c6f None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:28.862483 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb96ef5-ad72-4aa5-acfa-376515f08054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:28.863243 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb96ef5-ad72-4aa5-acfa-376515f08054 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:28.863621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb96ef5-ad72-4aa5-acfa-376515f08054 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:28.864000 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb96ef5-ad72-4aa5-acfa-376515f08054 None admin] PATH_INFO: `/v3/services` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:28.865123 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb96ef5-ad72-4aa5-acfa-376515f08054 None admin] RBAC: Adding query filter params (type=image) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:28.865742 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb96ef5-ad72-4aa5-acfa-376515f08054 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:28.867130 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb96ef5-ad72-4aa5-acfa-376515f08054 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:28.872235 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 533/1060] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:39:28 2022] GET /identity/v3/services?type=image => generated 350 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:39:28.877809 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb96ef5-ad72-4aa5-acfa-376515f08054 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:28.901987 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:28.962594 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c2c7a3-f149-41c7-be98-3dc678516c6f None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:28.996196 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb96ef5-ad72-4aa5-acfa-376515f08054 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:28.999797 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0143b4bb-80ad-4d0f-a228-3a0dc692904b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:29.000440 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0143b4bb-80ad-4d0f-a228-3a0dc692904b None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:29.000715 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0143b4bb-80ad-4d0f-a228-3a0dc692904b None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:29.000980 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0143b4bb-80ad-4d0f-a228-3a0dc692904b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:29.012243 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0143b4bb-80ad-4d0f-a228-3a0dc692904b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:39:29.017106 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-0143b4bb-80ad-4d0f-a228-3a0dc692904b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 12:39:29.017106 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:29.017106 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:29.017106 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:29.017106 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:29.017106 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:29.017106 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:29.017106 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:29.017106 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:29.017106 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:29.017106 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:29.017106 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:29.017106 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:39:29.017106 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:39:29.017106 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:39:29.017106 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:39:29.018221 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:39:29.018221 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:39:29.018221 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:39:29.018221 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:39:29.018221 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:29.018221 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:29.018221 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:29.018221 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:29.018221 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:29.018221 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:29.018221 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:29.018221 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:29.018221 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:29.018221 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:29.018221 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:29.018221 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:29.019075 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:29.019075 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:29.019075 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:29.019075 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:29.019075 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:39:29.019075 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:39:29.019075 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:39:29.019075 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:39:29.019075 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:39:29.019075 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:39:29.019075 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 12:39:29.019075 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:29.019075 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 528/1061] 10.222.0.20 () {56 vars in 1120 bytes} [Tue Jul 26 12:39:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:39:29.024137 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0143b4bb-80ad-4d0f-a228-3a0dc692904b None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:29.045265 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620bd0e1-b682-4ba3-b73e-9f7e5fce84f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:29.046022 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bd0e1-b682-4ba3-b73e-9f7e5fce84f1 None admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:29.046398 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bd0e1-b682-4ba3-b73e-9f7e5fce84f1 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:29.046787 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bd0e1-b682-4ba3-b73e-9f7e5fce84f1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:29.047590 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620bd0e1-b682-4ba3-b73e-9f7e5fce84f1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:29.049311 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620bd0e1-b682-4ba3-b73e-9f7e5fce84f1 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:29.061619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:29.079679 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 534/1062] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:39:28 2022] POST /identity/v3/endpoints => generated 327 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:39:29.148780 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0143b4bb-80ad-4d0f-a228-3a0dc692904b None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:29.206953 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf79602-4b82-4b2e-a177-a820fc6cf227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:29.207895 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf79602-4b82-4b2e-a177-a820fc6cf227 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:29.208395 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf79602-4b82-4b2e-a177-a820fc6cf227 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:29.208818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf79602-4b82-4b2e-a177-a820fc6cf227 None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:29.210680 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf79602-4b82-4b2e-a177-a820fc6cf227 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:29.211153 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf79602-4b82-4b2e-a177-a820fc6cf227 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:29.212930 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf79602-4b82-4b2e-a177-a820fc6cf227 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:29.223011 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 529/1063] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Jul 26 12:39:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:39:29.229537 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620bd0e1-b682-4ba3-b73e-9f7e5fce84f1 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:29.306805 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:29.341266 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620bd0e1-b682-4ba3-b73e-9f7e5fce84f1 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:29.376425 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f511b-4ef4-45de-bb02-10d9f7c9e5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:29.377062 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f511b-4ef4-45de-bb02-10d9f7c9e5a2 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:29.377338 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f511b-4ef4-45de-bb02-10d9f7c9e5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:29.377606 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f511b-4ef4-45de-bb02-10d9f7c9e5a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:29.389030 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0f511b-4ef4-45de-bb02-10d9f7c9e5a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:29.390502 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0f511b-4ef4-45de-bb02-10d9f7c9e5a2 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:29.406279 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-fc0f511b-4ef4-45de-bb02-10d9f7c9e5a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:39:29.406279 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:29.406279 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:29.406279 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:29.406279 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:29.406279 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:29.406279 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:29.406279 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:29.406279 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:29.406279 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:29.406279 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:29.406279 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:29.406279 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:39:29.406279 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:39:29.406279 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:39:29.407260 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:39:29.407260 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:29.407260 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:29.407260 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:29.407260 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:29.407260 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:29.407260 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:29.407260 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:29.407260 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:29.407260 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:29.407260 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:29.407260 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:29.407260 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:29.407260 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:29.407260 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:29.407260 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:29.407260 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:29.410203 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:39:29.410203 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:39:29.410203 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:39:29.410203 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:29.410203 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 535/1064] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:39:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:39:29.414905 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf79602-4b82-4b2e-a177-a820fc6cf227 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:29.511780 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:29.522386 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf79602-4b82-4b2e-a177-a820fc6cf227 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:29.556025 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae5c71-61c4-4ff3-b599-00f1428fb887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:29.556636 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae5c71-61c4-4ff3-b599-00f1428fb887 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:29.556920 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae5c71-61c4-4ff3-b599-00f1428fb887 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:29.557188 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae5c71-61c4-4ff3-b599-00f1428fb887 None admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:29.557860 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ae5c71-61c4-4ff3-b599-00f1428fb887 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:29.558159 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ae5c71-61c4-4ff3-b599-00f1428fb887 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:29.559500 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ae5c71-61c4-4ff3-b599-00f1428fb887 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:29.575733 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 530/1065] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:39:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:29.577456 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f511b-4ef4-45de-bb02-10d9f7c9e5a2 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:29.689205 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f511b-4ef4-45de-bb02-10d9f7c9e5a2 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:29.714457 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:29.729856 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee1f5c9-0df6-4996-b3de-d48a3bf2d6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:29.730525 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee1f5c9-0df6-4996-b3de-d48a3bf2d6d7 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:29.730807 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee1f5c9-0df6-4996-b3de-d48a3bf2d6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:29.731081 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee1f5c9-0df6-4996-b3de-d48a3bf2d6d7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:29.742817 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee1f5c9-0df6-4996-b3de-d48a3bf2d6d7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:29.745317 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee1f5c9-0df6-4996-b3de-d48a3bf2d6d7 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:29.760147 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-8ee1f5c9-0df6-4996-b3de-d48a3bf2d6d7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 12:39:29.760147 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:29.760147 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:29.760147 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:29.760147 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:29.760147 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:29.760147 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:29.760147 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:29.760147 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:29.760147 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:29.760147 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:29.760147 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:29.760147 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 12:39:29.760147 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 12:39:29.760147 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 12:39:29.760147 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 12:39:29.761688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:29.761688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:29.761688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 12:39:29.761688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:39:29.761688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 12:39:29.761688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:39:29.761688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:29.761688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:29.761688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:29.761688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:29.761688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:29.761688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:29.761688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:29.761688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:29.761688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:29.761688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:29.761688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:29.761688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:29.762745 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:29.762745 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:29.762745 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 12:39:29.762745 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 12:39:29.762745 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 12:39:29.762745 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 12:39:29.762745 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 12:39:29.762745 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 12:39:29.762745 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 12:39:29.762745 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:29.764228 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 536/1066] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Jul 26 12:39:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:39:29.772591 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae5c71-61c4-4ff3-b599-00f1428fb887 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:29.884799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae5c71-61c4-4ff3-b599-00f1428fb887 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:29.921335 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63dea00-38a8-452f-8335-8855074278b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:29.921955 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63dea00-38a8-452f-8335-8855074278b3 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:29.922234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63dea00-38a8-452f-8335-8855074278b3 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:29.922506 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63dea00-38a8-452f-8335-8855074278b3 None admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:29.923297 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63dea00-38a8-452f-8335-8855074278b3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:29.923602 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63dea00-38a8-452f-8335-8855074278b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:29.925049 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63dea00-38a8-452f-8335-8855074278b3 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:29.942930 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 531/1067] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Jul 26 12:39:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:29.948600 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee1f5c9-0df6-4996-b3de-d48a3bf2d6d7 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:30.049928 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee1f5c9-0df6-4996-b3de-d48a3bf2d6d7 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:30.089811 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db40bbb4-caba-4001-b0c7-5a4397f27702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:30.090468 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40bbb4-caba-4001-b0c7-5a4397f27702 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:30.090748 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40bbb4-caba-4001-b0c7-5a4397f27702 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:30.091090 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40bbb4-caba-4001-b0c7-5a4397f27702 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:30.104693 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db40bbb4-caba-4001-b0c7-5a4397f27702 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:30.106161 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db40bbb4-caba-4001-b0c7-5a4397f27702 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:30.126606 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-db40bbb4-caba-4001-b0c7-5a4397f27702 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:39:30.126606 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:30.126606 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:30.126606 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:30.126606 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:30.126606 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:30.126606 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:30.126606 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:30.126606 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:30.126606 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:30.126606 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:30.126606 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:30.126606 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:39:30.126606 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:39:30.126606 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:39:30.127563 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:39:30.127563 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:30.127563 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:30.127563 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:30.127563 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:30.127563 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:30.127563 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:30.127563 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:30.127563 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:30.127563 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:30.127563 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:30.127563 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:30.127563 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:30.127563 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:30.127563 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:30.127563 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:30.127563 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:30.128525 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:39:30.128525 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:39:30.128525 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:39:30.128525 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:30.129344 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 537/1068] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:39:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:39:30.136964 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63dea00-38a8-452f-8335-8855074278b3 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:30.150286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:30.246867 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63dea00-38a8-452f-8335-8855074278b3 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:30.248796 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d3b76-d491-4ba6-a68c-c643f4643459 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:30.249095 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d3b76-d491-4ba6-a68c-c643f4643459 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:30.249373 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d3b76-d491-4ba6-a68c-c643f4643459 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:30.252077 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 538/1069] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:39:30.257845 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8fee24-bfac-4c9f-919c-db1d26e4d521 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:30.258205 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8fee24-bfac-4c9f-919c-db1d26e4d521 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:30.258981 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8fee24-bfac-4c9f-919c-db1d26e4d521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:30.278475 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-ba8fee24-bfac-4c9f-919c-db1d26e4d521 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:30.279706 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-ba8fee24-bfac-4c9f-919c-db1d26e4d521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:30.285778 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a146ab7e-ff35-4cf7-968d-3000ba54c0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:30.287002 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146ab7e-ff35-4cf7-968d-3000ba54c0b4 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:30.287281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146ab7e-ff35-4cf7-968d-3000ba54c0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:30.287550 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146ab7e-ff35-4cf7-968d-3000ba54c0b4 None admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:30.288671 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a146ab7e-ff35-4cf7-968d-3000ba54c0b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:30.288973 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a146ab7e-ff35-4cf7-968d-3000ba54c0b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:30.290680 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a146ab7e-ff35-4cf7-968d-3000ba54c0b4 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:30.307797 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 532/1070] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:39:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:30.308812 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a146ab7e-ff35-4cf7-968d-3000ba54c0b4 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:30.315482 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:30.359317 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 539/1071] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 12:39:30.381385 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:30.382810 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8fee24-bfac-4c9f-919c-db1d26e4d521 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:30.416518 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a146ab7e-ff35-4cf7-968d-3000ba54c0b4 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:30.460652 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e1638-c7ff-446a-8727-3cb7fc00acdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:30.461288 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e1638-c7ff-446a-8727-3cb7fc00acdc None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:30.461565 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e1638-c7ff-446a-8727-3cb7fc00acdc None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:30.461833 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e1638-c7ff-446a-8727-3cb7fc00acdc None admin] PATH_INFO: `/v3/projects/service` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:30.468630 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74e1638-c7ff-446a-8727-3cb7fc00acdc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:30.470069 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74e1638-c7ff-446a-8727-3cb7fc00acdc None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:30.482818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-d74e1638-c7ff-446a-8727-3cb7fc00acdc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:39:30.482818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:30.482818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:30.482818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:30.482818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:30.482818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:30.482818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:30.482818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:30.482818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:30.482818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:30.482818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:30.482818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:30.482818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 12:39:30.482818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 12:39:30.482818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 12:39:30.483773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 12:39:30.483773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:30.483773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:30.483773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:30.483773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:30.483773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:30.483773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:30.483773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:30.483773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:30.483773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:30.483773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:30.483773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:30.483773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:30.483773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:30.483773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:30.483773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:30.483773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:30.484691 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 12:39:30.484691 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 12:39:30.484691 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 12:39:30.484691 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 12:39:30.484691 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 12:39:30.484691 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 12:39:30.484691 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:39:30.484691 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:30.484691 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 533/1072] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:39:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:39:30.490299 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e1638-c7ff-446a-8727-3cb7fc00acdc None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:30.495363 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8fee24-bfac-4c9f-919c-db1d26e4d521 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:30.532274 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d35dc59-d6a3-4aba-816f-11425274bce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:30.532932 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d35dc59-d6a3-4aba-816f-11425274bce6 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:30.533212 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d35dc59-d6a3-4aba-816f-11425274bce6 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:30.533485 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d35dc59-d6a3-4aba-816f-11425274bce6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:30.545618 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d35dc59-d6a3-4aba-816f-11425274bce6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:39:30.550563 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-9d35dc59-d6a3-4aba-816f-11425274bce6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:39:30.550563 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:30.550563 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:30.550563 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:30.550563 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:30.550563 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:30.550563 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:30.550563 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:30.550563 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:30.550563 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:30.550563 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:30.550563 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:30.550563 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:39:30.550563 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:39:30.550563 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:39:30.550563 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:39:30.551507 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:39:30.551507 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:39:30.551507 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:39:30.551507 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:39:30.551507 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:30.551507 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:30.551507 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:30.551507 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:30.551507 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:30.551507 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:30.551507 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:30.551507 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:30.551507 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:30.551507 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:30.551507 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:30.551507 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:30.553399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:30.553399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:30.553399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:30.553399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:30.553399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:39:30.553399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:39:30.553399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:39:30.553399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:39:30.553399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:39:30.553399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:39:30.553399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:39:30.553399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:30.553399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 540/1073] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 26 12:39:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:39:30.557521 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d35dc59-d6a3-4aba-816f-11425274bce6 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:30.594603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:30.602192 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e1638-c7ff-446a-8727-3cb7fc00acdc None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:30.632493 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:30.640271 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc405f-6855-45ee-8c45-b98060542aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:30.640895 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc405f-6855-45ee-8c45-b98060542aeb None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:30.641170 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc405f-6855-45ee-8c45-b98060542aeb None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:30.641441 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc405f-6855-45ee-8c45-b98060542aeb None admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:30.642175 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adc405f-6855-45ee-8c45-b98060542aeb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:30.642474 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adc405f-6855-45ee-8c45-b98060542aeb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:30.643906 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adc405f-6855-45ee-8c45-b98060542aeb None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:30.657156 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 534/1074] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 26 12:39:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:39:30.662895 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc405f-6855-45ee-8c45-b98060542aeb None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:30.675116 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d35dc59-d6a3-4aba-816f-11425274bce6 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:30.711007 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18104931-7327-4626-a0c3-6c163a1f5f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:30.711907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18104931-7327-4626-a0c3-6c163a1f5f7b None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:30.712352 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18104931-7327-4626-a0c3-6c163a1f5f7b None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:30.712624 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18104931-7327-4626-a0c3-6c163a1f5f7b None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:30.713349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18104931-7327-4626-a0c3-6c163a1f5f7b None admin] RBAC: Adding query filter params (name=admin) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:30.713662 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18104931-7327-4626-a0c3-6c163a1f5f7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:30.715100 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18104931-7327-4626-a0c3-6c163a1f5f7b None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:30.729865 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 541/1075] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:39:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:30.730913 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18104931-7327-4626-a0c3-6c163a1f5f7b None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:30.772680 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc405f-6855-45ee-8c45-b98060542aeb None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:30.807761 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8e44c2-6277-432f-af51-7fca3e2377f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:30.808428 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e44c2-6277-432f-af51-7fca3e2377f5 None admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:30.808705 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e44c2-6277-432f-af51-7fca3e2377f5 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:30.808973 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e44c2-6277-432f-af51-7fca3e2377f5 None admin] PATH_INFO: `/v3/projects/644278e7e8934b37a1e7ab20d35764cd/users/0f5d6f3c12244c0faf26048513dc49e7/roles/3655f7c95a7643d296b2c6a9cca87b4b` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:30.811141 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8e44c2-6277-432f-af51-7fca3e2377f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=644278e7e8934b37a1e7ab20d35764cd, user_id=0f5d6f3c12244c0faf26048513dc49e7, role_id=3655f7c95a7643d296b2c6a9cca87b4b)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:30.812593 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8e44c2-6277-432f-af51-7fca3e2377f5 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:30.827017 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 535/1076] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 26 12:39:30 2022] PUT /identity/v3/projects/644278e7e8934b37a1e7ab20d35764cd/users/0f5d6f3c12244c0faf26048513dc49e7/roles/3655f7c95a7643d296b2c6a9cca87b4b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:39:30.834723 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:30.851601 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18104931-7327-4626-a0c3-6c163a1f5f7b None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:30.892814 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab77dad-d777-4b07-8709-52a4d3e8a3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:30.893501 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab77dad-d777-4b07-8709-52a4d3e8a3ae None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:30.893850 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab77dad-d777-4b07-8709-52a4d3e8a3ae None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:30.894136 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab77dad-d777-4b07-8709-52a4d3e8a3ae None admin] PATH_INFO: `/v3/users/3bcd4efbdb0d4da09af979ef3023fb7e` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:30.895477 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab77dad-d777-4b07-8709-52a4d3e8a3ae None admin] RBAC: Authorizing `identity:get_user(user_id=3bcd4efbdb0d4da09af979ef3023fb7e)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:30.897617 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab77dad-d777-4b07-8709-52a4d3e8a3ae None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:30.904363 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 542/1077] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:39:30 2022] GET /identity/v3/users/3bcd4efbdb0d4da09af979ef3023fb7e => generated 285 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:30.905630 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8e44c2-6277-432f-af51-7fca3e2377f5 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:30.963325 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:31.019924 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8e44c2-6277-432f-af51-7fca3e2377f5 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:31.055286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e06a8-536f-45c6-a213-9da5fed1a847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:31.055963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e06a8-536f-45c6-a213-9da5fed1a847 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:31.056242 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e06a8-536f-45c6-a213-9da5fed1a847 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:31.056512 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e06a8-536f-45c6-a213-9da5fed1a847 None admin] PATH_INFO: `/v3/projects/149fa0d3b3664a1ebcd7741ec11f0256` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:31.057662 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e06a8-536f-45c6-a213-9da5fed1a847 None admin] RBAC: Authorizing `identity:get_project(project_id=149fa0d3b3664a1ebcd7741ec11f0256)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:31.059081 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e06a8-536f-45c6-a213-9da5fed1a847 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:31.060930 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 536/1078] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:39:30 2022] GET /identity/v3/projects/149fa0d3b3664a1ebcd7741ec11f0256 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:31.066731 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab77dad-d777-4b07-8709-52a4d3e8a3ae None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:31.172419 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f801c6e-8bc9-4e53-9631-c73353fb413a None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:31.172718 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f801c6e-8bc9-4e53-9631-c73353fb413a None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:31.172889 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f801c6e-8bc9-4e53-9631-c73353fb413a None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:31.174373 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 537/1079] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 12:39:31.175239 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab77dad-d777-4b07-8709-52a4d3e8a3ae None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:31.180873 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a94633-736a-4324-915f-cff4e3ba12e9 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:31.181157 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a94633-736a-4324-915f-cff4e3ba12e9 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:31.181436 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a94633-736a-4324-915f-cff4e3ba12e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:31.203473 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-25a94633-736a-4324-915f-cff4e3ba12e9 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:31.205693 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-25a94633-736a-4324-915f-cff4e3ba12e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:31.215823 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fbcf60-5e01-45cf-9a4e-db6d10d75a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:31.216498 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fbcf60-5e01-45cf-9a4e-db6d10d75a58 None admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:31.216777 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fbcf60-5e01-45cf-9a4e-db6d10d75a58 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:31.217050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fbcf60-5e01-45cf-9a4e-db6d10d75a58 None admin] PATH_INFO: `/v3/projects/149fa0d3b3664a1ebcd7741ec11f0256/users/3bcd4efbdb0d4da09af979ef3023fb7e/roles/0e78444e43c744668178971af4087ca0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:31.219007 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fbcf60-5e01-45cf-9a4e-db6d10d75a58 None admin] RBAC: Authorizing `identity:create_grant(project_id=149fa0d3b3664a1ebcd7741ec11f0256, user_id=3bcd4efbdb0d4da09af979ef3023fb7e, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:31.220468 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fbcf60-5e01-45cf-9a4e-db6d10d75a58 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:31.233869 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 543/1080] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 26 12:39:31 2022] PUT /identity/v3/projects/149fa0d3b3664a1ebcd7741ec11f0256/users/3bcd4efbdb0d4da09af979ef3023fb7e/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:39:31.287101 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 538/1081] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 12:39:31.309443 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fbcf60-5e01-45cf-9a4e-db6d10d75a58 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:31.330628 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:31.424768 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fbcf60-5e01-45cf-9a4e-db6d10d75a58 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:31.460661 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59b2df3-0791-45c1-9110-83820270c732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:31.461332 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b2df3-0791-45c1-9110-83820270c732 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:31.461616 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b2df3-0791-45c1-9110-83820270c732 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:31.461889 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b2df3-0791-45c1-9110-83820270c732 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:31.533173 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59b2df3-0791-45c1-9110-83820270c732 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:31.534631 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59b2df3-0791-45c1-9110-83820270c732 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:31.552687 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:31.643678 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 544/1082] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:39:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 12:39:31.644903 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a94633-736a-4324-915f-cff4e3ba12e9 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:31.698745 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:31.767694 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a94633-736a-4324-915f-cff4e3ba12e9 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:31.805284 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02793387-5553-4b1f-9e5e-790006f95867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:31.806202 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02793387-5553-4b1f-9e5e-790006f95867 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:31.806587 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02793387-5553-4b1f-9e5e-790006f95867 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:31.806964 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02793387-5553-4b1f-9e5e-790006f95867 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:31.817438 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02793387-5553-4b1f-9e5e-790006f95867 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:31.818999 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02793387-5553-4b1f-9e5e-790006f95867 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:31.834596 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-02793387-5553-4b1f-9e5e-790006f95867 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:39:31.834596 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:31.834596 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:31.834596 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:31.834596 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:31.834596 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:31.834596 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:31.834596 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:31.834596 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:31.834596 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:31.834596 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:31.834596 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:31.834596 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:39:31.834596 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:39:31.834596 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:39:31.835719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:39:31.835719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:31.835719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:31.835719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:31.835719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:31.835719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:31.835719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:31.835719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:31.835719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:31.835719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:31.835719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:31.835719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:31.835719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:31.835719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:31.835719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:31.835719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:31.835719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:31.836835 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:39:31.836835 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:39:31.836835 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:39:31.836835 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:31.838504 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 539/1083] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:39:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:39:31.846519 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59b2df3-0791-45c1-9110-83820270c732 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:31.954290 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59b2df3-0791-45c1-9110-83820270c732 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:31.972612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:31.994506 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab47ff-743c-44bf-be98-e54f97ac1ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:31.995218 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab47ff-743c-44bf-be98-e54f97ac1ec7 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:31.995570 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab47ff-743c-44bf-be98-e54f97ac1ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:31.995934 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab47ff-743c-44bf-be98-e54f97ac1ec7 None admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:31.996703 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab47ff-743c-44bf-be98-e54f97ac1ec7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:31.997080 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab47ff-743c-44bf-be98-e54f97ac1ec7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:31.998631 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab47ff-743c-44bf-be98-e54f97ac1ec7 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:32.011001 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 545/1084] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:39:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:32.871228 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f8fc7-c684-4ecc-bc16-bb8341702da6 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:32.871809 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f8fc7-c684-4ecc-bc16-bb8341702da6 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:32.871809 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f8fc7-c684-4ecc-bc16-bb8341702da6 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:32.873220 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 540/1085] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:39:32.879177 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f2a77-2b9d-43af-88a8-e27fa8719169 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:32.879612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f2a77-2b9d-43af-88a8-e27fa8719169 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:32.880392 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f2a77-2b9d-43af-88a8-e27fa8719169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:32.900483 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-485f2a77-2b9d-43af-88a8-e27fa8719169 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:32.901691 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-485f2a77-2b9d-43af-88a8-e27fa8719169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:32.969599 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 546/1086] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:39:32.993274 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f8fc7-c684-4ecc-bc16-bb8341702da6 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:33.090898 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:33.095511 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f8fc7-c684-4ecc-bc16-bb8341702da6 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:33.129640 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee12e5ee-1e7c-4ef8-ba3a-86fbbd7f53df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:33.130789 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee12e5ee-1e7c-4ef8-ba3a-86fbbd7f53df None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:33.131063 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee12e5ee-1e7c-4ef8-ba3a-86fbbd7f53df None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:33.131331 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee12e5ee-1e7c-4ef8-ba3a-86fbbd7f53df None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:33.141265 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee12e5ee-1e7c-4ef8-ba3a-86fbbd7f53df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:39:33.146218 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-ee12e5ee-1e7c-4ef8-ba3a-86fbbd7f53df None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 12:39:33.146218 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:33.146218 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:33.146218 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:33.146218 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:33.146218 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:33.146218 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:33.146218 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:33.146218 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:33.146218 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:33.146218 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:33.146218 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:33.146218 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:39:33.146218 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:39:33.146218 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:39:33.146218 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:39:33.147164 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:39:33.147164 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:39:33.147164 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:39:33.147164 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:39:33.147164 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:33.147164 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:33.147164 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:33.147164 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:33.147164 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:33.147164 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:33.147164 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:33.147164 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:33.147164 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:33.147164 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:33.147164 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:33.147164 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:33.148050 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:33.148050 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:33.148050 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:33.148050 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:33.148050 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:39:33.148050 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:39:33.148050 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:39:33.148050 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:39:33.148050 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:39:33.148050 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:39:33.148050 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 12:39:33.148050 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:33.148050 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 541/1087] 10.222.0.20 () {56 vars in 1120 bytes} [Tue Jul 26 12:39:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:39:33.153482 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f2a77-2b9d-43af-88a8-e27fa8719169 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:33.245829 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f81b6d-a411-4c77-9303-5fc7c338c280 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:33.246148 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f81b6d-a411-4c77-9303-5fc7c338c280 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:33.246426 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f81b6d-a411-4c77-9303-5fc7c338c280 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:33.247836 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 542/1088] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 12:39:33.253449 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23518f-4231-4874-8ca0-2a53bd1a70e6 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:33.253741 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23518f-4231-4874-8ca0-2a53bd1a70e6 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:33.254016 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23518f-4231-4874-8ca0-2a53bd1a70e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:33.257440 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f2a77-2b9d-43af-88a8-e27fa8719169 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:33.264920 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:33.272509 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-ca23518f-4231-4874-8ca0-2a53bd1a70e6 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:33.273781 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-ca23518f-4231-4874-8ca0-2a53bd1a70e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:33.293406 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6b935-f6c4-4875-9685-9f232962f9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:33.294017 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6b935-f6c4-4875-9685-9f232962f9f0 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:33.294299 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6b935-f6c4-4875-9685-9f232962f9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:33.294571 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6b935-f6c4-4875-9685-9f232962f9f0 None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:33.295286 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a6b935-f6c4-4875-9685-9f232962f9f0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:33.295587 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a6b935-f6c4-4875-9685-9f232962f9f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:33.296983 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a6b935-f6c4-4875-9685-9f232962f9f0 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:33.305049 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 547/1089] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Jul 26 12:39:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:39:33.310839 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6b935-f6c4-4875-9685-9f232962f9f0 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:33.341143 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 543/1090] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 12:39:33.364843 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca23518f-4231-4874-8ca0-2a53bd1a70e6 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:33.415187 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6b935-f6c4-4875-9685-9f232962f9f0 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:33.447256 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc2e1b1-fa7d-4c71-962e-f952050bdf61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:33.447919 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc2e1b1-fa7d-4c71-962e-f952050bdf61 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:33.448196 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc2e1b1-fa7d-4c71-962e-f952050bdf61 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:33.448465 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc2e1b1-fa7d-4c71-962e-f952050bdf61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:33.452598 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:33.459718 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc2e1b1-fa7d-4c71-962e-f952050bdf61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:33.461178 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc2e1b1-fa7d-4c71-962e-f952050bdf61 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:33.462929 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca23518f-4231-4874-8ca0-2a53bd1a70e6 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:33.475851 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-fbc2e1b1-fa7d-4c71-962e-f952050bdf61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:39:33.475851 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:33.475851 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:33.475851 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:33.475851 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:33.475851 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:33.475851 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:33.475851 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:33.475851 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:33.475851 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:33.475851 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:33.475851 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:33.475851 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:39:33.475851 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:39:33.475851 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:39:33.476805 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:39:33.476805 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:33.476805 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:33.476805 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:33.476805 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:33.476805 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:33.476805 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:33.476805 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:33.476805 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:33.476805 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:33.476805 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:33.476805 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:33.476805 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:33.476805 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:33.476805 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:33.476805 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:33.476805 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:33.477749 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:39:33.477749 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:39:33.477749 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:39:33.477749 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:33.477749 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 548/1091] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:39:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:39:33.482802 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc2e1b1-fa7d-4c71-962e-f952050bdf61 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:33.498588 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c124bf-0be9-428c-92b4-ae9bf0726d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:33.499318 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c124bf-0be9-428c-92b4-ae9bf0726d20 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:33.499603 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c124bf-0be9-428c-92b4-ae9bf0726d20 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:33.499912 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c124bf-0be9-428c-92b4-ae9bf0726d20 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:33.505878 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:33.511539 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c124bf-0be9-428c-92b4-ae9bf0726d20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:39:33.516520 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-64c124bf-0be9-428c-92b4-ae9bf0726d20 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:39:33.516520 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:33.516520 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:33.516520 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:33.516520 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:33.516520 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:33.516520 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:33.516520 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:33.516520 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:33.516520 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:33.516520 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:33.516520 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:33.516520 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:39:33.516520 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:39:33.516520 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:39:33.516520 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:39:33.517642 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:39:33.517642 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:39:33.517642 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:39:33.517642 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:39:33.517642 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:33.517642 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:33.517642 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:33.517642 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:33.517642 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:33.517642 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:33.517642 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:33.517642 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:33.517642 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:33.517642 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:33.517642 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:33.517642 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:33.518662 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:33.518662 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:33.518662 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:33.518662 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:33.518662 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:39:33.518662 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:39:33.518662 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:39:33.518662 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:39:33.518662 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:39:33.518662 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:39:33.518662 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:39:33.518662 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:33.518662 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 544/1092] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 26 12:39:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:39:33.524432 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c124bf-0be9-428c-92b4-ae9bf0726d20 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:33.586319 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc2e1b1-fa7d-4c71-962e-f952050bdf61 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:33.619429 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c215324-cf69-4193-8a09-777f4da6a896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:33.620178 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c215324-cf69-4193-8a09-777f4da6a896 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:33.620351 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c215324-cf69-4193-8a09-777f4da6a896 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:33.620627 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c215324-cf69-4193-8a09-777f4da6a896 None admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:33.621315 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c215324-cf69-4193-8a09-777f4da6a896 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:33.621627 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c215324-cf69-4193-8a09-777f4da6a896 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:33.622996 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c215324-cf69-4193-8a09-777f4da6a896 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:33.623596 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c124bf-0be9-428c-92b4-ae9bf0726d20 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:33.635262 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 549/1093] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:39:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:39:33.640768 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c215324-cf69-4193-8a09-777f4da6a896 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:33.658923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa2871-e924-4852-ba0f-c05a32514775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:33.659540 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa2871-e924-4852-ba0f-c05a32514775 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:33.659851 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa2871-e924-4852-ba0f-c05a32514775 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:33.660128 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa2871-e924-4852-ba0f-c05a32514775 None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:33.660839 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fa2871-e924-4852-ba0f-c05a32514775 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:33.661177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fa2871-e924-4852-ba0f-c05a32514775 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:33.662604 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fa2871-e924-4852-ba0f-c05a32514775 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:33.670965 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 545/1094] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:39:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:39:33.676932 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa2871-e924-4852-ba0f-c05a32514775 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:33.691709 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:33.722631 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:33.742323 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c215324-cf69-4193-8a09-777f4da6a896 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:33.775999 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8917bb1e-99da-4237-8d71-ca256d24c82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:33.776660 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917bb1e-99da-4237-8d71-ca256d24c82e None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:33.776947 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917bb1e-99da-4237-8d71-ca256d24c82e None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:33.777219 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917bb1e-99da-4237-8d71-ca256d24c82e None admin] PATH_INFO: `/v3/users/nova` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:33.780997 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa2871-e924-4852-ba0f-c05a32514775 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:33.784399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8917bb1e-99da-4237-8d71-ca256d24c82e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:33.785847 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8917bb1e-99da-4237-8d71-ca256d24c82e None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:33.798011 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-8917bb1e-99da-4237-8d71-ca256d24c82e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 12:39:33.798011 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:33.798011 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:33.798011 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:33.798011 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:33.798011 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:33.798011 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:33.798011 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:33.798011 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:33.798011 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:33.798011 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:33.798011 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:33.798011 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 12:39:33.798011 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 12:39:33.798011 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 12:39:33.798011 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 12:39:33.798961 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:33.798961 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:33.798961 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 12:39:33.798961 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:39:33.798961 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 12:39:33.798961 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:39:33.798961 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:33.798961 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:33.798961 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:33.798961 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:33.798961 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:33.798961 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:33.798961 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:33.798961 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:33.798961 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:33.798961 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:33.798961 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:33.798961 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:33.800161 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:33.800161 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:33.800161 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 12:39:33.800161 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 12:39:33.800161 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 12:39:33.800161 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 12:39:33.800161 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 12:39:33.800161 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 12:39:33.800161 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 12:39:33.800161 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:33.800161 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 550/1095] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Jul 26 12:39:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:39:33.805502 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8917bb1e-99da-4237-8d71-ca256d24c82e None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:33.816558 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b07e22-5150-4f33-8345-acdb316c947a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:33.817194 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b07e22-5150-4f33-8345-acdb316c947a None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:33.818008 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b07e22-5150-4f33-8345-acdb316c947a None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:33.818278 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b07e22-5150-4f33-8345-acdb316c947a None admin] PATH_INFO: `/v3/users/3bcd4efbdb0d4da09af979ef3023fb7e` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:33.819458 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b07e22-5150-4f33-8345-acdb316c947a None admin] RBAC: Authorizing `identity:get_user(user_id=3bcd4efbdb0d4da09af979ef3023fb7e)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:33.820928 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b07e22-5150-4f33-8345-acdb316c947a None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:33.822531 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 546/1096] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:39:33 2022] GET /identity/v3/users/3bcd4efbdb0d4da09af979ef3023fb7e => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:39:33.828297 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b07e22-5150-4f33-8345-acdb316c947a None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:33.908763 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8917bb1e-99da-4237-8d71-ca256d24c82e None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:33.909358 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:33.934511 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b07e22-5150-4f33-8345-acdb316c947a None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:33.950082 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e59d8e1-a1de-4360-859e-1eb558284291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:33.950691 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e59d8e1-a1de-4360-859e-1eb558284291 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:33.950971 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e59d8e1-a1de-4360-859e-1eb558284291 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:33.951244 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e59d8e1-a1de-4360-859e-1eb558284291 None admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:33.952063 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e59d8e1-a1de-4360-859e-1eb558284291 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:33.952363 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e59d8e1-a1de-4360-859e-1eb558284291 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:33.953798 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e59d8e1-a1de-4360-859e-1eb558284291 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:33.963486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:33.971069 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29a862a-2418-459b-875a-5ef2549a45ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:33.971256 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 551/1097] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Jul 26 12:39:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:39:33.971805 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a862a-2418-459b-875a-5ef2549a45ac None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:33.972160 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a862a-2418-459b-875a-5ef2549a45ac None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:33.972421 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a862a-2418-459b-875a-5ef2549a45ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:33.976848 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e59d8e1-a1de-4360-859e-1eb558284291 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:34.040583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29a862a-2418-459b-875a-5ef2549a45ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:34.042022 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29a862a-2418-459b-875a-5ef2549a45ac None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:34.075513 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e59d8e1-a1de-4360-859e-1eb558284291 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:34.109325 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c61b2c-72fe-4d2c-993b-ed7648b76981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:34.109942 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c61b2c-72fe-4d2c-993b-ed7648b76981 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:34.110225 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c61b2c-72fe-4d2c-993b-ed7648b76981 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:34.110497 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c61b2c-72fe-4d2c-993b-ed7648b76981 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:34.121296 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:34.141646 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 547/1098] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:39:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 26 12:39:34.147899 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29a862a-2418-459b-875a-5ef2549a45ac None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:34.180917 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:34.185015 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c61b2c-72fe-4d2c-993b-ed7648b76981 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:34.186410 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c61b2c-72fe-4d2c-993b-ed7648b76981 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:34.251065 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29a862a-2418-459b-875a-5ef2549a45ac None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:34.285412 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 552/1099] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:39:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 26 12:39:34.287398 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f167de-725a-4d6e-988b-65967deecf23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:34.288061 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f167de-725a-4d6e-988b-65967deecf23 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:34.288337 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f167de-725a-4d6e-988b-65967deecf23 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:34.288608 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f167de-725a-4d6e-988b-65967deecf23 None admin] PATH_INFO: `/v3/projects/149fa0d3b3664a1ebcd7741ec11f0256` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:34.289740 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f167de-725a-4d6e-988b-65967deecf23 None admin] RBAC: Authorizing `identity:get_project(project_id=149fa0d3b3664a1ebcd7741ec11f0256)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:34.291152 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c61b2c-72fe-4d2c-993b-ed7648b76981 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:34.291253 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f167de-725a-4d6e-988b-65967deecf23 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:34.297709 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 548/1100] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:39:34 2022] GET /identity/v3/projects/149fa0d3b3664a1ebcd7741ec11f0256 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:39:34.298819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f167de-725a-4d6e-988b-65967deecf23 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:34.347946 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:34.386684 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:34.390518 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c61b2c-72fe-4d2c-993b-ed7648b76981 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:34.401877 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f167de-725a-4d6e-988b-65967deecf23 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:34.425649 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a872abdf-5482-45e6-a50f-7139502d76c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:34.426287 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872abdf-5482-45e6-a50f-7139502d76c6 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:34.426566 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872abdf-5482-45e6-a50f-7139502d76c6 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:34.426897 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872abdf-5482-45e6-a50f-7139502d76c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:34.434477 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4686952a-4a92-420e-8a5f-001fd2e469f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:34.435127 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686952a-4a92-420e-8a5f-001fd2e469f2 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:34.435345 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686952a-4a92-420e-8a5f-001fd2e469f2 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:34.435611 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686952a-4a92-420e-8a5f-001fd2e469f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:34.436418 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4686952a-4a92-420e-8a5f-001fd2e469f2 None admin] RBAC: Adding query filter params (role.id=0e78444e43c744668178971af4087ca0, scope.project.id=149fa0d3b3664a1ebcd7741ec11f0256, user.id=3bcd4efbdb0d4da09af979ef3023fb7e) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:34.436720 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4686952a-4a92-420e-8a5f-001fd2e469f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:34.436813 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a872abdf-5482-45e6-a50f-7139502d76c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:34.438149 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4686952a-4a92-420e-8a5f-001fd2e469f2 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:34.438241 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a872abdf-5482-45e6-a50f-7139502d76c6 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:34.444412 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 549/1101] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:39:34 2022] GET /identity/v3/role_assignments?user.id=3bcd4efbdb0d4da09af979ef3023fb7e&scope.project.id=149fa0d3b3664a1ebcd7741ec11f0256&role.id=0e78444e43c744668178971af4087ca0 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:39:34.454710 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-a872abdf-5482-45e6-a50f-7139502d76c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:39:34.454710 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:34.454710 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:34.454710 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:34.454710 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:34.454710 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:34.454710 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:34.454710 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:34.454710 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:34.454710 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:34.454710 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:34.454710 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:34.454710 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:39:34.454710 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:39:34.454710 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:39:34.455937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:39:34.455937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:34.455937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:34.455937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:34.455937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:34.455937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:34.455937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:34.455937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:34.455937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:34.455937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:34.455937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:34.455937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:34.455937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:34.455937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:34.455937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:34.455937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:34.455937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:34.456898 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:39:34.456898 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:39:34.456898 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:39:34.456898 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:34.456898 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 553/1102] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:39:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:39:34.463254 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4686952a-4a92-420e-8a5f-001fd2e469f2 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:34.571408 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4686952a-4a92-420e-8a5f-001fd2e469f2 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:34.585861 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:34.609108 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e159ad1-cb8d-47b5-9168-4cf509a979d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:34.609757 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e159ad1-cb8d-47b5-9168-4cf509a979d3 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:34.610025 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e159ad1-cb8d-47b5-9168-4cf509a979d3 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:34.610292 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e159ad1-cb8d-47b5-9168-4cf509a979d3 None admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:34.610978 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e159ad1-cb8d-47b5-9168-4cf509a979d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:34.611286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e159ad1-cb8d-47b5-9168-4cf509a979d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:34.612686 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e159ad1-cb8d-47b5-9168-4cf509a979d3 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:34.626437 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 550/1103] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:39:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:34.632885 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a872abdf-5482-45e6-a50f-7139502d76c6 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:34.734035 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a872abdf-5482-45e6-a50f-7139502d76c6 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:34.767127 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab3593-d3a7-49e0-96fd-8a0453a4854a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:34.767797 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab3593-d3a7-49e0-96fd-8a0453a4854a None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:34.768080 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab3593-d3a7-49e0-96fd-8a0453a4854a None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:34.768432 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab3593-d3a7-49e0-96fd-8a0453a4854a None admin] PATH_INFO: `/v3/projects/service` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:34.775584 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab3593-d3a7-49e0-96fd-8a0453a4854a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:34.777056 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab3593-d3a7-49e0-96fd-8a0453a4854a None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:34.789034 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-5fab3593-d3a7-49e0-96fd-8a0453a4854a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:39:34.789034 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:34.789034 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:34.789034 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:34.789034 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:34.789034 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:34.789034 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:34.789034 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:34.789034 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:34.789034 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:34.789034 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:34.789034 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:34.789034 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 12:39:34.789034 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 12:39:34.789034 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 12:39:34.790385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 12:39:34.790385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:34.790385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:34.790385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:34.790385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:34.790385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:34.790385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:34.790385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:34.790385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:34.790385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:34.790385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:34.790385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:34.790385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:34.790385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:34.790385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:34.790385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:34.790385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:34.791327 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 12:39:34.791327 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 12:39:34.791327 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 12:39:34.791327 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 12:39:34.791327 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 12:39:34.791327 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 12:39:34.791327 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:39:34.791327 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:34.791327 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 554/1104] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:39:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:39:34.796201 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e159ad1-cb8d-47b5-9168-4cf509a979d3 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:34.901634 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e159ad1-cb8d-47b5-9168-4cf509a979d3 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:34.937042 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2437644b-f5f6-4f22-9361-6d11f5544932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:34.937670 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2437644b-f5f6-4f22-9361-6d11f5544932 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:34.937929 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2437644b-f5f6-4f22-9361-6d11f5544932 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:34.938204 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2437644b-f5f6-4f22-9361-6d11f5544932 None admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:34.938948 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2437644b-f5f6-4f22-9361-6d11f5544932 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:34.939266 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2437644b-f5f6-4f22-9361-6d11f5544932 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:34.940720 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2437644b-f5f6-4f22-9361-6d11f5544932 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:34.952957 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 551/1105] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 26 12:39:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:34.959256 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab3593-d3a7-49e0-96fd-8a0453a4854a None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:34.972397 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:35.061446 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab3593-d3a7-49e0-96fd-8a0453a4854a None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:35.093937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da37699f-ccd9-455a-999c-8ae19396b681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:35.094538 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37699f-ccd9-455a-999c-8ae19396b681 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:35.094822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37699f-ccd9-455a-999c-8ae19396b681 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:35.095100 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37699f-ccd9-455a-999c-8ae19396b681 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:35.095921 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da37699f-ccd9-455a-999c-8ae19396b681 None admin] RBAC: Adding query filter params (role.id=3655f7c95a7643d296b2c6a9cca87b4b, scope.project.id=644278e7e8934b37a1e7ab20d35764cd, user.id=0f5d6f3c12244c0faf26048513dc49e7) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:35.096233 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da37699f-ccd9-455a-999c-8ae19396b681 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:35.097665 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da37699f-ccd9-455a-999c-8ae19396b681 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:35.104402 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 555/1106] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:39:34 2022] GET /identity/v3/role_assignments?user.id=0f5d6f3c12244c0faf26048513dc49e7&scope.project.id=644278e7e8934b37a1e7ab20d35764cd&role.id=3655f7c95a7643d296b2c6a9cca87b4b => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:36.484732 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb086cb-52a9-48bf-a36e-a6ee638ac80d None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:36.485276 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb086cb-52a9-48bf-a36e-a6ee638ac80d None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:36.485276 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb086cb-52a9-48bf-a36e-a6ee638ac80d None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:36.486642 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 552/1107] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:39:36.492013 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d64f1-0dcb-471b-8595-970a26a34be8 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:36.492392 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d64f1-0dcb-471b-8595-970a26a34be8 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:36.492570 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d64f1-0dcb-471b-8595-970a26a34be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:36.511229 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-603d64f1-0dcb-471b-8595-970a26a34be8 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:36.512454 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-603d64f1-0dcb-471b-8595-970a26a34be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:36.577075 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 556/1108] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:39:36.601896 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb086cb-52a9-48bf-a36e-a6ee638ac80d None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:36.633609 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:36.702473 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb086cb-52a9-48bf-a36e-a6ee638ac80d None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:36.737135 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd0714-ede8-4082-b8c5-21b7b20c443e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:36.737692 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd0714-ede8-4082-b8c5-21b7b20c443e None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:36.737973 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd0714-ede8-4082-b8c5-21b7b20c443e None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:36.738243 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd0714-ede8-4082-b8c5-21b7b20c443e None admin] PATH_INFO: `/v3/domains/default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:36.739404 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dd0714-ede8-4082-b8c5-21b7b20c443e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:36.741122 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dd0714-ede8-4082-b8c5-21b7b20c443e None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:36.742704 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 553/1109] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:39:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:36.748533 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603d64f1-0dcb-471b-8595-970a26a34be8 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:36.751591 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:36.848752 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603d64f1-0dcb-471b-8595-970a26a34be8 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:36.882414 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05985d44-9c32-4220-9d72-b8151a2ce269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:36.883019 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05985d44-9c32-4220-9d72-b8151a2ce269 None admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:36.883295 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05985d44-9c32-4220-9d72-b8151a2ce269 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:36.883565 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05985d44-9c32-4220-9d72-b8151a2ce269 None admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:36.884263 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05985d44-9c32-4220-9d72-b8151a2ce269 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:36.885684 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05985d44-9c32-4220-9d72-b8151a2ce269 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:36.906200 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 557/1110] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Jul 26 12:39:36 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:39:38.937740 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533bec0b-58e4-4556-a6c1-b4ae18dd35a4 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:38.938331 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533bec0b-58e4-4556-a6c1-b4ae18dd35a4 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:38.938331 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533bec0b-58e4-4556-a6c1-b4ae18dd35a4 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:38.939627 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 554/1111] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:39:38.944875 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2711e-0328-46fc-b3b9-975a7d6e25c0 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:38.945275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2711e-0328-46fc-b3b9-975a7d6e25c0 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:38.945439 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2711e-0328-46fc-b3b9-975a7d6e25c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:38.963107 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-c1d2711e-0328-46fc-b3b9-975a7d6e25c0 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:38.964341 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-c1d2711e-0328-46fc-b3b9-975a7d6e25c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:39.019811 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:39.031945 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 558/1112] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:39:39.055872 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533bec0b-58e4-4556-a6c1-b4ae18dd35a4 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:39.151764 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:39.157073 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533bec0b-58e4-4556-a6c1-b4ae18dd35a4 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:39.191107 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd96f45-4433-4c09-a6b0-0da0c48797d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:39.191715 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd96f45-4433-4c09-a6b0-0da0c48797d3 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:39.192002 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd96f45-4433-4c09-a6b0-0da0c48797d3 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:39.192281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd96f45-4433-4c09-a6b0-0da0c48797d3 None admin] PATH_INFO: `/v3/roles/6589779ddd834447a1ed1921e1c834f6` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:39.193720 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd96f45-4433-4c09-a6b0-0da0c48797d3 None admin] RBAC: Authorizing `identity:get_role(role_id=6589779ddd834447a1ed1921e1c834f6)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:39.195088 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd96f45-4433-4c09-a6b0-0da0c48797d3 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:39.196486 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 555/1113] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:39:39 2022] GET /identity/v3/roles/6589779ddd834447a1ed1921e1c834f6 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:39.202357 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d2711e-0328-46fc-b3b9-975a7d6e25c0 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:39.299029 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d2711e-0328-46fc-b3b9-975a7d6e25c0 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:39.331367 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46743989-5c4e-459a-aafe-9acf686a211e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:39.332026 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46743989-5c4e-459a-aafe-9acf686a211e None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:39.332305 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46743989-5c4e-459a-aafe-9acf686a211e None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:39.332582 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46743989-5c4e-459a-aafe-9acf686a211e None admin] PATH_INFO: `/v3/users/66b60db1b33b4c7a9801e7ad9cddff08` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:39.345749 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46743989-5c4e-459a-aafe-9acf686a211e None admin] RBAC: Authorizing `identity:get_user(user_id=66b60db1b33b4c7a9801e7ad9cddff08)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:39.347190 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46743989-5c4e-459a-aafe-9acf686a211e None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:39.348986 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 559/1114] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:39:39 2022] GET /identity/v3/users/66b60db1b33b4c7a9801e7ad9cddff08 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:39.354841 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd96f45-4433-4c09-a6b0-0da0c48797d3 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:39.453025 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd96f45-4433-4c09-a6b0-0da0c48797d3 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:39.486010 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a20ee-3151-4cb1-96d1-f0d61f373b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:39.486615 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a20ee-3151-4cb1-96d1-f0d61f373b95 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:39.486896 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a20ee-3151-4cb1-96d1-f0d61f373b95 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:39.487168 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a20ee-3151-4cb1-96d1-f0d61f373b95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:39.521138 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:39.553939 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a20ee-3151-4cb1-96d1-f0d61f373b95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:39.555381 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a20ee-3151-4cb1-96d1-f0d61f373b95 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:39.650968 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 556/1115] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:39:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 12:39:39.657075 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46743989-5c4e-459a-aafe-9acf686a211e None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:39.705570 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:39.754453 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46743989-5c4e-459a-aafe-9acf686a211e None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:39.789250 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dba312-2cf9-4bab-bc7a-5614e799b77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:39.789877 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dba312-2cf9-4bab-bc7a-5614e799b77e None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:39.790153 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dba312-2cf9-4bab-bc7a-5614e799b77e None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:39.790423 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dba312-2cf9-4bab-bc7a-5614e799b77e None admin] PATH_INFO: `/v3/projects/149fa0d3b3664a1ebcd7741ec11f0256` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:39.791550 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dba312-2cf9-4bab-bc7a-5614e799b77e None admin] RBAC: Authorizing `identity:get_project(project_id=149fa0d3b3664a1ebcd7741ec11f0256)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:39.793012 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dba312-2cf9-4bab-bc7a-5614e799b77e None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:39.794676 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 560/1116] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:39:39 2022] GET /identity/v3/projects/149fa0d3b3664a1ebcd7741ec11f0256 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:39.802372 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a20ee-3151-4cb1-96d1-f0d61f373b95 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:39.874714 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:39.908745 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a20ee-3151-4cb1-96d1-f0d61f373b95 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:39.942895 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88f0273-df9d-4125-afd0-4d407b0919f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:39.943534 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f0273-df9d-4125-afd0-4d407b0919f6 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:39.943809 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f0273-df9d-4125-afd0-4d407b0919f6 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:39.944085 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f0273-df9d-4125-afd0-4d407b0919f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:39.944864 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88f0273-df9d-4125-afd0-4d407b0919f6 None admin] RBAC: Adding query filter params (role.id=6589779ddd834447a1ed1921e1c834f6, scope.project.id=149fa0d3b3664a1ebcd7741ec11f0256, user.id=66b60db1b33b4c7a9801e7ad9cddff08) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:39.945173 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88f0273-df9d-4125-afd0-4d407b0919f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:39.946602 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88f0273-df9d-4125-afd0-4d407b0919f6 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:39.952033 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 557/1117] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:39:39 2022] GET /identity/v3/role_assignments?user.id=66b60db1b33b4c7a9801e7ad9cddff08&scope.project.id=149fa0d3b3664a1ebcd7741ec11f0256&role.id=6589779ddd834447a1ed1921e1c834f6 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:41.890457 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56ffa4-7955-4b29-8f23-b3d3c0edc502 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:41.891027 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56ffa4-7955-4b29-8f23-b3d3c0edc502 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:41.891027 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56ffa4-7955-4b29-8f23-b3d3c0edc502 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:41.892399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 561/1118] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:39:41.898409 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb45ef7-eedc-4c24-bb19-1d764ef0e38f None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:41.898897 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb45ef7-eedc-4c24-bb19-1d764ef0e38f None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:41.899023 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb45ef7-eedc-4c24-bb19-1d764ef0e38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:41.917663 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-2eb45ef7-eedc-4c24-bb19-1d764ef0e38f None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:41.918838 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-2eb45ef7-eedc-4c24-bb19-1d764ef0e38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:41.983769 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 558/1119] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:39:42.006697 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56ffa4-7955-4b29-8f23-b3d3c0edc502 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:42.104822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56ffa4-7955-4b29-8f23-b3d3c0edc502 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:42.121345 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:42.138427 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ede3474-3203-4665-8ef6-0587448981e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:42.139057 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede3474-3203-4665-8ef6-0587448981e6 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:42.139331 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede3474-3203-4665-8ef6-0587448981e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:42.139600 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede3474-3203-4665-8ef6-0587448981e6 None admin] PATH_INFO: `/v3/roles/6589779ddd834447a1ed1921e1c834f6` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:42.141150 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ede3474-3203-4665-8ef6-0587448981e6 None admin] RBAC: Authorizing `identity:get_role(role_id=6589779ddd834447a1ed1921e1c834f6)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:42.142467 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ede3474-3203-4665-8ef6-0587448981e6 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:42.143830 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 562/1120] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:39:42 2022] GET /identity/v3/roles/6589779ddd834447a1ed1921e1c834f6 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:42.150136 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb45ef7-eedc-4c24-bb19-1d764ef0e38f None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:42.209792 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:42.250660 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb45ef7-eedc-4c24-bb19-1d764ef0e38f None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:42.293477 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d2b8bd-ee81-4a6c-a65f-ecb01c8976a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:42.294097 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d2b8bd-ee81-4a6c-a65f-ecb01c8976a1 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:42.294375 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d2b8bd-ee81-4a6c-a65f-ecb01c8976a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:42.294647 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d2b8bd-ee81-4a6c-a65f-ecb01c8976a1 None admin] PATH_INFO: `/v3/users/66b60db1b33b4c7a9801e7ad9cddff08` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:42.295814 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d2b8bd-ee81-4a6c-a65f-ecb01c8976a1 None admin] RBAC: Authorizing `identity:get_user(user_id=66b60db1b33b4c7a9801e7ad9cddff08)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:42.297246 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d2b8bd-ee81-4a6c-a65f-ecb01c8976a1 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:42.298842 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 559/1121] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:39:42 2022] GET /identity/v3/users/66b60db1b33b4c7a9801e7ad9cddff08 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:42.305361 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ede3474-3203-4665-8ef6-0587448981e6 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:42.402767 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ede3474-3203-4665-8ef6-0587448981e6 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:42.435775 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40812ab9-92ec-4a65-826a-71cab00b60c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:42.436400 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40812ab9-92ec-4a65-826a-71cab00b60c6 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:42.436682 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40812ab9-92ec-4a65-826a-71cab00b60c6 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:42.436958 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40812ab9-92ec-4a65-826a-71cab00b60c6 None admin] PATH_INFO: `/v3/projects/149fa0d3b3664a1ebcd7741ec11f0256` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:42.438107 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40812ab9-92ec-4a65-826a-71cab00b60c6 None admin] RBAC: Authorizing `identity:get_project(project_id=149fa0d3b3664a1ebcd7741ec11f0256)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:42.439535 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40812ab9-92ec-4a65-826a-71cab00b60c6 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:42.441219 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 563/1122] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:39:42 2022] GET /identity/v3/projects/149fa0d3b3664a1ebcd7741ec11f0256 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:42.447223 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d2b8bd-ee81-4a6c-a65f-ecb01c8976a1 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:42.542828 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d2b8bd-ee81-4a6c-a65f-ecb01c8976a1 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:42.570090 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:42.576185 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ceffcd-8879-4925-aeb7-fe885ca24799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:42.576854 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ceffcd-8879-4925-aeb7-fe885ca24799 None admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:42.577134 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ceffcd-8879-4925-aeb7-fe885ca24799 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:42.577404 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ceffcd-8879-4925-aeb7-fe885ca24799 None admin] PATH_INFO: `/v3/projects/149fa0d3b3664a1ebcd7741ec11f0256/users/66b60db1b33b4c7a9801e7ad9cddff08/roles/6589779ddd834447a1ed1921e1c834f6` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:42.579561 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ceffcd-8879-4925-aeb7-fe885ca24799 None admin] RBAC: Authorizing `identity:create_grant(project_id=149fa0d3b3664a1ebcd7741ec11f0256, user_id=66b60db1b33b4c7a9801e7ad9cddff08, role_id=6589779ddd834447a1ed1921e1c834f6)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:42.581032 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ceffcd-8879-4925-aeb7-fe885ca24799 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:42.594151 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 560/1123] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 26 12:39:42 2022] PUT /identity/v3/projects/149fa0d3b3664a1ebcd7741ec11f0256/users/66b60db1b33b4c7a9801e7ad9cddff08/roles/6589779ddd834447a1ed1921e1c834f6 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:39:44.611998 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d88e9-d71d-4e15-aa98-f13031fd6d34 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:44.612551 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d88e9-d71d-4e15-aa98-f13031fd6d34 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:44.612551 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d88e9-d71d-4e15-aa98-f13031fd6d34 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:44.613912 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 564/1124] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:39:44.620282 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a753f-7121-4a9d-9a8b-e25bde1d2a9f None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:44.620802 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a753f-7121-4a9d-9a8b-e25bde1d2a9f None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:44.620908 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a753f-7121-4a9d-9a8b-e25bde1d2a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:44.639338 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-2b2a753f-7121-4a9d-9a8b-e25bde1d2a9f None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:44.640620 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-2b2a753f-7121-4a9d-9a8b-e25bde1d2a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:44.707280 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 561/1125] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:39:44.731385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9d88e9-d71d-4e15-aa98-f13031fd6d34 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:44.787521 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:44.837459 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9d88e9-d71d-4e15-aa98-f13031fd6d34 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:44.873435 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c314be32-0708-455a-ab31-b26e1dbf8da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:44.873990 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314be32-0708-455a-ab31-b26e1dbf8da5 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:44.874263 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314be32-0708-455a-ab31-b26e1dbf8da5 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:44.874533 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314be32-0708-455a-ab31-b26e1dbf8da5 None admin] PATH_INFO: `/v3/roles/6589779ddd834447a1ed1921e1c834f6` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:44.876027 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c314be32-0708-455a-ab31-b26e1dbf8da5 None admin] RBAC: Authorizing `identity:get_role(role_id=6589779ddd834447a1ed1921e1c834f6)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:44.877408 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c314be32-0708-455a-ab31-b26e1dbf8da5 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:44.878768 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 565/1126] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:39:44 2022] GET /identity/v3/roles/6589779ddd834447a1ed1921e1c834f6 => generated 223 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:44.885324 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a753f-7121-4a9d-9a8b-e25bde1d2a9f None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:44.987251 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a753f-7121-4a9d-9a8b-e25bde1d2a9f None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:45.007861 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:45.022623 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db8fda7-5583-43f2-806e-f9ec3aa55b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:45.023249 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8fda7-5583-43f2-806e-f9ec3aa55b84 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:45.023534 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8fda7-5583-43f2-806e-f9ec3aa55b84 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:45.023842 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8fda7-5583-43f2-806e-f9ec3aa55b84 None admin] PATH_INFO: `/v3/users/66b60db1b33b4c7a9801e7ad9cddff08` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:45.024987 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db8fda7-5583-43f2-806e-f9ec3aa55b84 None admin] RBAC: Authorizing `identity:get_user(user_id=66b60db1b33b4c7a9801e7ad9cddff08)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:45.026421 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db8fda7-5583-43f2-806e-f9ec3aa55b84 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:45.028047 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 562/1127] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:39:44 2022] GET /identity/v3/users/66b60db1b33b4c7a9801e7ad9cddff08 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:45.033868 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c314be32-0708-455a-ab31-b26e1dbf8da5 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:45.132483 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c314be32-0708-455a-ab31-b26e1dbf8da5 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:45.153204 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:45.167240 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc20ea-f9c9-421f-8b79-2431c4b89657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:45.167875 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc20ea-f9c9-421f-8b79-2431c4b89657 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:45.168153 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc20ea-f9c9-421f-8b79-2431c4b89657 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:45.168425 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc20ea-f9c9-421f-8b79-2431c4b89657 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:45.246492 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfc20ea-f9c9-421f-8b79-2431c4b89657 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:45.247971 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfc20ea-f9c9-421f-8b79-2431c4b89657 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:45.342763 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 566/1128] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:39:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 12:39:45.348691 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db8fda7-5583-43f2-806e-f9ec3aa55b84 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:45.446607 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db8fda7-5583-43f2-806e-f9ec3aa55b84 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:45.479916 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c54eb22-f470-4f46-88b4-6ed05516afde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:45.480556 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54eb22-f470-4f46-88b4-6ed05516afde None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:45.480839 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54eb22-f470-4f46-88b4-6ed05516afde None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:45.481116 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54eb22-f470-4f46-88b4-6ed05516afde None admin] PATH_INFO: `/v3/projects/149fa0d3b3664a1ebcd7741ec11f0256` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:45.482261 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c54eb22-f470-4f46-88b4-6ed05516afde None admin] RBAC: Authorizing `identity:get_project(project_id=149fa0d3b3664a1ebcd7741ec11f0256)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:45.483722 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c54eb22-f470-4f46-88b4-6ed05516afde None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:45.485411 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 563/1129] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:39:45 2022] GET /identity/v3/projects/149fa0d3b3664a1ebcd7741ec11f0256 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:45.492201 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc20ea-f9c9-421f-8b79-2431c4b89657 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:45.516344 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:45.622283 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc20ea-f9c9-421f-8b79-2431c4b89657 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:45.655718 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dd639f-b1d9-4b49-8854-057dd8cc7486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:45.656314 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd639f-b1d9-4b49-8854-057dd8cc7486 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:45.656592 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd639f-b1d9-4b49-8854-057dd8cc7486 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:45.656864 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd639f-b1d9-4b49-8854-057dd8cc7486 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:45.657637 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dd639f-b1d9-4b49-8854-057dd8cc7486 None admin] RBAC: Adding query filter params (role.id=6589779ddd834447a1ed1921e1c834f6, scope.project.id=149fa0d3b3664a1ebcd7741ec11f0256, user.id=66b60db1b33b4c7a9801e7ad9cddff08) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:45.657941 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dd639f-b1d9-4b49-8854-057dd8cc7486 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:45.659367 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dd639f-b1d9-4b49-8854-057dd8cc7486 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:45.664989 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 567/1130] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:39:45 2022] GET /identity/v3/role_assignments?user.id=66b60db1b33b4c7a9801e7ad9cddff08&scope.project.id=149fa0d3b3664a1ebcd7741ec11f0256&role.id=6589779ddd834447a1ed1921e1c834f6 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:47.657378 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2933de6-a380-4b5f-b6bc-1488ca6c8abd None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:47.657908 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2933de6-a380-4b5f-b6bc-1488ca6c8abd None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:47.657908 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2933de6-a380-4b5f-b6bc-1488ca6c8abd None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:47.659270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 564/1131] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:39:47.664440 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c60856-7396-47a3-8c54-b99c91b5e66e None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:47.664871 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c60856-7396-47a3-8c54-b99c91b5e66e None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:47.664997 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c60856-7396-47a3-8c54-b99c91b5e66e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:47.683291 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-f3c60856-7396-47a3-8c54-b99c91b5e66e None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:47.684516 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-f3c60856-7396-47a3-8c54-b99c91b5e66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:47.750327 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 568/1132] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:39:47.774297 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2933de6-a380-4b5f-b6bc-1488ca6c8abd None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:47.829639 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:47.877648 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2933de6-a380-4b5f-b6bc-1488ca6c8abd None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:47.911960 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5b93b2-d3f8-423d-88df-9e044f64c073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:47.912517 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b93b2-d3f8-423d-88df-9e044f64c073 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:47.912795 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b93b2-d3f8-423d-88df-9e044f64c073 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:47.913163 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b93b2-d3f8-423d-88df-9e044f64c073 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:47.915080 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5b93b2-d3f8-423d-88df-9e044f64c073 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:47.917808 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5b93b2-d3f8-423d-88df-9e044f64c073 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:47.919826 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 565/1133] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:39:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:47.926866 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c60856-7396-47a3-8c54-b99c91b5e66e None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:47.943532 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:48.027456 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c60856-7396-47a3-8c54-b99c91b5e66e None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:48.059937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ac2b3-9a4f-43be-81bc-d729613b4c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:48.060543 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ac2b3-9a4f-43be-81bc-d729613b4c60 None admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:48.060824 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ac2b3-9a4f-43be-81bc-d729613b4c60 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:48.061096 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ac2b3-9a4f-43be-81bc-d729613b4c60 None admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:48.061779 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38ac2b3-9a4f-43be-81bc-d729613b4c60 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:48.063210 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38ac2b3-9a4f-43be-81bc-d729613b4c60 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:48.090095 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 569/1134] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Jul 26 12:39:47 2022] POST /identity/v3/projects => generated 311 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:39:50.084296 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44445a12-7c30-44e5-b088-c7254c29e2bf None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:50.084856 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44445a12-7c30-44e5-b088-c7254c29e2bf None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:50.084856 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44445a12-7c30-44e5-b088-c7254c29e2bf None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:50.086170 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 566/1135] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:39:50.091250 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598d7b6-fe11-4872-a55c-40ff155d8521 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:50.091683 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598d7b6-fe11-4872-a55c-40ff155d8521 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:50.091836 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598d7b6-fe11-4872-a55c-40ff155d8521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:50.110190 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-4598d7b6-fe11-4872-a55c-40ff155d8521 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:50.111373 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-4598d7b6-fe11-4872-a55c-40ff155d8521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:50.170338 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:50.179473 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 570/1136] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:39:50.203137 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44445a12-7c30-44e5-b088-c7254c29e2bf None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:50.305792 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44445a12-7c30-44e5-b088-c7254c29e2bf None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:50.328491 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:50.339442 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c5ac0-9bfd-4459-9b12-0f7827b54ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:50.340041 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c5ac0-9bfd-4459-9b12-0f7827b54ce8 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:50.340325 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c5ac0-9bfd-4459-9b12-0f7827b54ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:50.340598 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c5ac0-9bfd-4459-9b12-0f7827b54ce8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:50.341743 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c5ac0-9bfd-4459-9b12-0f7827b54ce8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:50.343181 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c5ac0-9bfd-4459-9b12-0f7827b54ce8 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:50.344798 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 567/1137] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:39:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:50.351249 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4598d7b6-fe11-4872-a55c-40ff155d8521 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:50.447043 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4598d7b6-fe11-4872-a55c-40ff155d8521 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:50.480333 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09560475-2512-4df6-a351-4b1bce94810e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:50.480934 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09560475-2512-4df6-a351-4b1bce94810e None admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:50.481208 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09560475-2512-4df6-a351-4b1bce94810e None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:50.481477 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09560475-2512-4df6-a351-4b1bce94810e None admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:50.482148 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09560475-2512-4df6-a351-4b1bce94810e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:50.483563 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09560475-2512-4df6-a351-4b1bce94810e None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:50.503166 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 571/1138] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Jul 26 12:39:50 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:39:52.562264 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7366d95-ae29-4042-bc50-b2159a7f0efa None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:52.562797 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7366d95-ae29-4042-bc50-b2159a7f0efa None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:52.562892 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7366d95-ae29-4042-bc50-b2159a7f0efa None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:52.564253 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 568/1139] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:39:52.569331 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759df9c3-1609-4f25-826e-0d1eeadc37e2 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:52.569741 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759df9c3-1609-4f25-826e-0d1eeadc37e2 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:52.569889 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759df9c3-1609-4f25-826e-0d1eeadc37e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:52.587555 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-759df9c3-1609-4f25-826e-0d1eeadc37e2 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:52.588769 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-759df9c3-1609-4f25-826e-0d1eeadc37e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:52.635553 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:52.656370 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 572/1140] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:39:52.679825 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7366d95-ae29-4042-bc50-b2159a7f0efa None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:52.778108 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7366d95-ae29-4042-bc50-b2159a7f0efa None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:52.811140 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389fe4fb-c63c-4dc8-ad0c-5daaae0970af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:52.811795 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389fe4fb-c63c-4dc8-ad0c-5daaae0970af None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:52.812079 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389fe4fb-c63c-4dc8-ad0c-5daaae0970af None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:52.812406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389fe4fb-c63c-4dc8-ad0c-5daaae0970af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:52.822382 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389fe4fb-c63c-4dc8-ad0c-5daaae0970af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:39:52.827268 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-389fe4fb-c63c-4dc8-ad0c-5daaae0970af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:39:52.827268 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:52.827268 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:52.827268 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:52.827268 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:52.827268 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:52.827268 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:52.827268 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:52.827268 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:52.827268 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:52.827268 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:52.827268 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:52.827268 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:39:52.827268 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:39:52.827268 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:39:52.827268 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:39:52.828509 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:39:52.828509 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:39:52.828509 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:39:52.828509 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:39:52.828509 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:52.828509 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:52.828509 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:52.828509 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:52.828509 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:52.828509 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:52.828509 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:52.828509 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:52.828509 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:52.828509 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:52.828509 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:52.828509 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:52.829427 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:52.829427 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:52.829427 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:52.829427 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:52.829427 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:39:52.829427 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:39:52.829427 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:39:52.829427 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:39:52.829427 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:39:52.829427 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:39:52.829427 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:39:52.829427 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:52.829427 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 569/1141] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 26 12:39:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:39:52.835113 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759df9c3-1609-4f25-826e-0d1eeadc37e2 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:52.935063 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759df9c3-1609-4f25-826e-0d1eeadc37e2 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:52.968222 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54328026-c0ab-425a-8192-a4d37309a967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:52.968834 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54328026-c0ab-425a-8192-a4d37309a967 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:52.969118 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54328026-c0ab-425a-8192-a4d37309a967 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:52.969395 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54328026-c0ab-425a-8192-a4d37309a967 None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:52.970114 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54328026-c0ab-425a-8192-a4d37309a967 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:52.970423 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54328026-c0ab-425a-8192-a4d37309a967 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:52.971834 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54328026-c0ab-425a-8192-a4d37309a967 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:52.980364 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 573/1142] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:39:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:52.986342 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389fe4fb-c63c-4dc8-ad0c-5daaae0970af None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:53.038851 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:53.085666 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389fe4fb-c63c-4dc8-ad0c-5daaae0970af None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:53.118614 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf5edbb-162a-4429-9a71-d828db0ab84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:53.119239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5edbb-162a-4429-9a71-d828db0ab84f None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:53.119521 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5edbb-162a-4429-9a71-d828db0ab84f None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:53.119821 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5edbb-162a-4429-9a71-d828db0ab84f None admin] PATH_INFO: `/v3/users/b9935aaae04c4d1aa17a03c76a885b68` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:53.133911 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf5edbb-162a-4429-9a71-d828db0ab84f None admin] RBAC: Authorizing `identity:get_user(user_id=b9935aaae04c4d1aa17a03c76a885b68)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:53.135341 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf5edbb-162a-4429-9a71-d828db0ab84f None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:53.137001 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 570/1143] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:39:52 2022] GET /identity/v3/users/b9935aaae04c4d1aa17a03c76a885b68 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:53.142986 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54328026-c0ab-425a-8192-a4d37309a967 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:53.188682 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:53.243218 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54328026-c0ab-425a-8192-a4d37309a967 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:53.276424 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460d7948-da5a-4f62-977a-b33e1640cc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:53.277030 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d7948-da5a-4f62-977a-b33e1640cc30 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:53.277312 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d7948-da5a-4f62-977a-b33e1640cc30 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:53.277589 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d7948-da5a-4f62-977a-b33e1640cc30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:53.343120 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460d7948-da5a-4f62-977a-b33e1640cc30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:53.344531 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460d7948-da5a-4f62-977a-b33e1640cc30 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:53.388142 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:53.441326 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 574/1144] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:39:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 12:39:53.447812 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf5edbb-162a-4429-9a71-d828db0ab84f None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:53.542079 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf5edbb-162a-4429-9a71-d828db0ab84f None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:53.552824 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:53.576190 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8797ec93-a1ba-43e7-8723-da21cfab6f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:53.576811 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797ec93-a1ba-43e7-8723-da21cfab6f99 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:53.577079 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797ec93-a1ba-43e7-8723-da21cfab6f99 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:53.577347 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797ec93-a1ba-43e7-8723-da21cfab6f99 None admin] PATH_INFO: `/v3/projects/3fb6372143554de5b3e60110aebe2a7a` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:53.578476 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8797ec93-a1ba-43e7-8723-da21cfab6f99 None admin] RBAC: Authorizing `identity:get_project(project_id=3fb6372143554de5b3e60110aebe2a7a)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:53.579914 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8797ec93-a1ba-43e7-8723-da21cfab6f99 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:53.581551 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 571/1145] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:39:53 2022] GET /identity/v3/projects/3fb6372143554de5b3e60110aebe2a7a => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:53.587819 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460d7948-da5a-4f62-977a-b33e1640cc30 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:53.683197 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460d7948-da5a-4f62-977a-b33e1640cc30 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:53.716805 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6362fb5-7e1a-4eb9-bc33-52b3eb126037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:53.717406 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6362fb5-7e1a-4eb9-bc33-52b3eb126037 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:53.717681 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6362fb5-7e1a-4eb9-bc33-52b3eb126037 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:53.717945 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6362fb5-7e1a-4eb9-bc33-52b3eb126037 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:53.718713 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6362fb5-7e1a-4eb9-bc33-52b3eb126037 None admin] RBAC: Adding query filter params (role.id=0e78444e43c744668178971af4087ca0, scope.project.id=3fb6372143554de5b3e60110aebe2a7a, user.id=b9935aaae04c4d1aa17a03c76a885b68) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:53.719015 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6362fb5-7e1a-4eb9-bc33-52b3eb126037 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:53.720463 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6362fb5-7e1a-4eb9-bc33-52b3eb126037 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:53.726283 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 575/1146] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:39:53 2022] GET /identity/v3/role_assignments?user.id=b9935aaae04c4d1aa17a03c76a885b68&scope.project.id=3fb6372143554de5b3e60110aebe2a7a&role.id=0e78444e43c744668178971af4087ca0 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:55.664125 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38281d22-79a7-42c1-b084-2d30e0435776 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:55.664670 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38281d22-79a7-42c1-b084-2d30e0435776 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:55.664670 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38281d22-79a7-42c1-b084-2d30e0435776 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:55.666027 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 572/1147] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:39:55.671016 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef56bd-a8b8-4344-9a97-78d7fa4bfc44 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:55.671456 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef56bd-a8b8-4344-9a97-78d7fa4bfc44 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:55.671548 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef56bd-a8b8-4344-9a97-78d7fa4bfc44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:55.689794 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-caef56bd-a8b8-4344-9a97-78d7fa4bfc44 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:55.690997 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-caef56bd-a8b8-4344-9a97-78d7fa4bfc44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:55.693672 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:55.758523 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 576/1148] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:39:55.781120 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38281d22-79a7-42c1-b084-2d30e0435776 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:55.886243 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38281d22-79a7-42c1-b084-2d30e0435776 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:55.925844 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4460589-16ba-4f72-bdc6-062e8f3e43d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:55.926412 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4460589-16ba-4f72-bdc6-062e8f3e43d0 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:55.926691 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4460589-16ba-4f72-bdc6-062e8f3e43d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:55.926998 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4460589-16ba-4f72-bdc6-062e8f3e43d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:55.936814 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4460589-16ba-4f72-bdc6-062e8f3e43d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:39:55.941725 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-c4460589-16ba-4f72-bdc6-062e8f3e43d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:39:55.941725 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:55.941725 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:55.941725 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:55.941725 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:55.941725 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:55.941725 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:55.941725 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:55.941725 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:55.941725 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:55.941725 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:55.941725 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:55.941725 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:39:55.941725 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:39:55.941725 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:39:55.941725 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:39:55.942901 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:39:55.942901 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:39:55.942901 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:39:55.942901 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:39:55.942901 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:55.942901 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:55.942901 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:55.942901 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:55.942901 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:55.942901 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:55.942901 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:55.942901 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:55.942901 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:55.942901 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:55.942901 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:55.942901 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:55.943940 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:55.943940 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:55.943940 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:55.943940 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:55.943940 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:39:55.943940 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:39:55.943940 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:39:55.943940 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:39:55.943940 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:39:55.943940 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:39:55.943940 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:39:55.943940 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:39:55.943940 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 573/1149] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 26 12:39:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:39:55.948959 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caef56bd-a8b8-4344-9a97-78d7fa4bfc44 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:56.046963 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caef56bd-a8b8-4344-9a97-78d7fa4bfc44 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:56.080186 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19efdea2-2512-4357-8b86-baa94019256b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:56.080794 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19efdea2-2512-4357-8b86-baa94019256b None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:56.081077 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19efdea2-2512-4357-8b86-baa94019256b None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:56.081355 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19efdea2-2512-4357-8b86-baa94019256b None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:56.082068 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19efdea2-2512-4357-8b86-baa94019256b None admin] RBAC: Adding query filter params (name=admin) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:56.082373 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19efdea2-2512-4357-8b86-baa94019256b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:56.083809 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19efdea2-2512-4357-8b86-baa94019256b None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:56.088918 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:56.096883 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 577/1150] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:39:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:56.098198 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4460589-16ba-4f72-bdc6-062e8f3e43d0 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:56.138059 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:56.197859 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4460589-16ba-4f72-bdc6-062e8f3e43d0 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:56.230677 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502c1eb-1e54-45c0-9df2-ca5d72859650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:56.231299 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502c1eb-1e54-45c0-9df2-ca5d72859650 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:56.231574 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502c1eb-1e54-45c0-9df2-ca5d72859650 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:56.232056 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502c1eb-1e54-45c0-9df2-ca5d72859650 None admin] PATH_INFO: `/v3/users/b9935aaae04c4d1aa17a03c76a885b68` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:56.233203 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f502c1eb-1e54-45c0-9df2-ca5d72859650 None admin] RBAC: Authorizing `identity:get_user(user_id=b9935aaae04c4d1aa17a03c76a885b68)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:56.234616 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f502c1eb-1e54-45c0-9df2-ca5d72859650 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:56.236239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 574/1151] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:39:56 2022] GET /identity/v3/users/b9935aaae04c4d1aa17a03c76a885b68 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:56.242380 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19efdea2-2512-4357-8b86-baa94019256b None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:56.337839 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19efdea2-2512-4357-8b86-baa94019256b None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:56.370420 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f53880-5d29-48b1-936c-6a9e617c4563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:56.371040 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f53880-5d29-48b1-936c-6a9e617c4563 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:56.371315 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f53880-5d29-48b1-936c-6a9e617c4563 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:56.371583 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f53880-5d29-48b1-936c-6a9e617c4563 None admin] PATH_INFO: `/v3/projects/3fb6372143554de5b3e60110aebe2a7a` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:56.372740 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f53880-5d29-48b1-936c-6a9e617c4563 None admin] RBAC: Authorizing `identity:get_project(project_id=3fb6372143554de5b3e60110aebe2a7a)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:56.374160 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f53880-5d29-48b1-936c-6a9e617c4563 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:56.375829 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 578/1152] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:39:56 2022] GET /identity/v3/projects/3fb6372143554de5b3e60110aebe2a7a => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:56.382458 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502c1eb-1e54-45c0-9df2-ca5d72859650 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:56.483489 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502c1eb-1e54-45c0-9df2-ca5d72859650 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:56.491428 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:56.518177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367b6f30-416a-4106-b23b-9eb5543f38d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:56.518830 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b6f30-416a-4106-b23b-9eb5543f38d9 None admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:56.519113 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b6f30-416a-4106-b23b-9eb5543f38d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:56.519389 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b6f30-416a-4106-b23b-9eb5543f38d9 None admin] PATH_INFO: `/v3/projects/3fb6372143554de5b3e60110aebe2a7a/users/b9935aaae04c4d1aa17a03c76a885b68/roles/0e78444e43c744668178971af4087ca0` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:56.521373 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367b6f30-416a-4106-b23b-9eb5543f38d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=3fb6372143554de5b3e60110aebe2a7a, user_id=b9935aaae04c4d1aa17a03c76a885b68, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:56.522809 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367b6f30-416a-4106-b23b-9eb5543f38d9 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:56.535109 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 575/1153] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 26 12:39:56 2022] PUT /identity/v3/projects/3fb6372143554de5b3e60110aebe2a7a/users/b9935aaae04c4d1aa17a03c76a885b68/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:39:58.528896 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25666f68-4d20-47f0-b2b0-9f48e2c87a0d None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:58.529426 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25666f68-4d20-47f0-b2b0-9f48e2c87a0d None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:58.529426 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25666f68-4d20-47f0-b2b0-9f48e2c87a0d None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:58.531179 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 579/1154] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:39:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:39:58.537127 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39afb42b-88b5-4a1e-8131-aea43552a6a5 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:58.537529 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39afb42b-88b5-4a1e-8131-aea43552a6a5 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:58.537671 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39afb42b-88b5-4a1e-8131-aea43552a6a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:58.556266 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-39afb42b-88b5-4a1e-8131-aea43552a6a5 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:39:58.557459 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-39afb42b-88b5-4a1e-8131-aea43552a6a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:39:58.623720 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 576/1155] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:39:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:39:58.647052 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25666f68-4d20-47f0-b2b0-9f48e2c87a0d None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:58.713737 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:58.748526 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25666f68-4d20-47f0-b2b0-9f48e2c87a0d None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:58.781357 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f6ba5c-3bc8-48c5-9394-6ab96c170749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:58.781912 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f6ba5c-3bc8-48c5-9394-6ab96c170749 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:58.782194 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f6ba5c-3bc8-48c5-9394-6ab96c170749 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:58.782468 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f6ba5c-3bc8-48c5-9394-6ab96c170749 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:58.792841 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f6ba5c-3bc8-48c5-9394-6ab96c170749 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:39:58.797714 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-47f6ba5c-3bc8-48c5-9394-6ab96c170749 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:39:58.797714 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:39:58.797714 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:39:58.797714 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:39:58.797714 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:39:58.797714 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:39:58.797714 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:39:58.797714 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:39:58.797714 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:39:58.797714 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:39:58.797714 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:39:58.797714 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:39:58.797714 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:39:58.797714 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:39:58.797714 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:39:58.797714 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:39:58.798859 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:39:58.798859 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:39:58.798859 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:39:58.798859 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:39:58.798859 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:39:58.798859 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:39:58.798859 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:39:58.798859 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:39:58.798859 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:39:58.798859 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:39:58.798859 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:39:58.798859 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:39:58.798859 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:39:58.798859 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:39:58.798859 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:39:58.798859 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:39:58.799935 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:39:58.799935 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:39:58.799935 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:39:58.799935 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:39:58.799935 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:39:58.799935 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:39:58.799935 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:39:58.799935 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:39:58.799935 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:39:58.799935 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:39:58.799935 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:39:58.799935 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:39:58.799935 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 580/1156] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 26 12:39:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:39:58.805510 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39afb42b-88b5-4a1e-8131-aea43552a6a5 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:58.897872 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:58.904228 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39afb42b-88b5-4a1e-8131-aea43552a6a5 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:58.938643 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44568a-65fe-410b-9aa2-c9ca7b15b81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:58.939260 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44568a-65fe-410b-9aa2-c9ca7b15b81e None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:58.939539 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44568a-65fe-410b-9aa2-c9ca7b15b81e None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:58.939840 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44568a-65fe-410b-9aa2-c9ca7b15b81e None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:58.940549 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af44568a-65fe-410b-9aa2-c9ca7b15b81e None admin] RBAC: Adding query filter params (name=admin) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:58.940855 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af44568a-65fe-410b-9aa2-c9ca7b15b81e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:58.942231 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af44568a-65fe-410b-9aa2-c9ca7b15b81e None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:58.950708 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 577/1157] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:39:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:58.956669 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f6ba5c-3bc8-48c5-9394-6ab96c170749 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:59.054670 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f6ba5c-3bc8-48c5-9394-6ab96c170749 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:59.079124 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:59.087302 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c6dca2-5333-4aa2-ab11-7fd1493e5f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:59.087963 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6dca2-5333-4aa2-ab11-7fd1493e5f90 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:59.088243 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6dca2-5333-4aa2-ab11-7fd1493e5f90 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:59.088514 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6dca2-5333-4aa2-ab11-7fd1493e5f90 None admin] PATH_INFO: `/v3/users/b9935aaae04c4d1aa17a03c76a885b68` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:59.089660 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c6dca2-5333-4aa2-ab11-7fd1493e5f90 None admin] RBAC: Authorizing `identity:get_user(user_id=b9935aaae04c4d1aa17a03c76a885b68)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:59.091088 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c6dca2-5333-4aa2-ab11-7fd1493e5f90 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:59.092704 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 581/1158] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:39:58 2022] GET /identity/v3/users/b9935aaae04c4d1aa17a03c76a885b68 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:59.099109 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44568a-65fe-410b-9aa2-c9ca7b15b81e None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:59.197261 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44568a-65fe-410b-9aa2-c9ca7b15b81e None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:59.231327 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf5ab38-79de-4fb7-89f5-a54551e7d022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:59.231975 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf5ab38-79de-4fb7-89f5-a54551e7d022 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:59.232265 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf5ab38-79de-4fb7-89f5-a54551e7d022 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:59.232539 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf5ab38-79de-4fb7-89f5-a54551e7d022 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:59.259617 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:59.299466 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf5ab38-79de-4fb7-89f5-a54551e7d022 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:59.300946 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf5ab38-79de-4fb7-89f5-a54551e7d022 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:59.397435 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 578/1159] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:39:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 12:39:59.403248 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c6dca2-5333-4aa2-ab11-7fd1493e5f90 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:59.499537 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c6dca2-5333-4aa2-ab11-7fd1493e5f90 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:59.531916 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2669d4e-6a5b-400d-9182-fdec44d7db8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:59.532537 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2669d4e-6a5b-400d-9182-fdec44d7db8f None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:59.532829 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2669d4e-6a5b-400d-9182-fdec44d7db8f None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:59.533098 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2669d4e-6a5b-400d-9182-fdec44d7db8f None admin] PATH_INFO: `/v3/projects/3fb6372143554de5b3e60110aebe2a7a` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:59.534229 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2669d4e-6a5b-400d-9182-fdec44d7db8f None admin] RBAC: Authorizing `identity:get_project(project_id=3fb6372143554de5b3e60110aebe2a7a)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:59.535655 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2669d4e-6a5b-400d-9182-fdec44d7db8f None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:59.537321 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 582/1160] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:39:59 2022] GET /identity/v3/projects/3fb6372143554de5b3e60110aebe2a7a => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:39:59.543429 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf5ab38-79de-4fb7-89f5-a54551e7d022 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:39:59.599740 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:39:59.644075 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf5ab38-79de-4fb7-89f5-a54551e7d022 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:39:59.681213 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eb5984-d87d-4d5b-925d-f5ce57fcf531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:39:59.681736 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb5984-d87d-4d5b-925d-f5ce57fcf531 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:39:59.682024 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb5984-d87d-4d5b-925d-f5ce57fcf531 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:39:59.682299 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb5984-d87d-4d5b-925d-f5ce57fcf531 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:39:59.683080 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eb5984-d87d-4d5b-925d-f5ce57fcf531 None admin] RBAC: Adding query filter params (role.id=0e78444e43c744668178971af4087ca0, scope.project.id=3fb6372143554de5b3e60110aebe2a7a, user.id=b9935aaae04c4d1aa17a03c76a885b68) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:39:59.683391 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eb5984-d87d-4d5b-925d-f5ce57fcf531 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:39:59.684856 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eb5984-d87d-4d5b-925d-f5ce57fcf531 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:39:59.690649 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 579/1161] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:39:59 2022] GET /identity/v3/role_assignments?user.id=b9935aaae04c4d1aa17a03c76a885b68&scope.project.id=3fb6372143554de5b3e60110aebe2a7a&role.id=0e78444e43c744668178971af4087ca0 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:40:01.705003 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cbabb8-9fc6-460b-be61-a59b842e4059 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:40:01.705574 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cbabb8-9fc6-460b-be61-a59b842e4059 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:40:01.705574 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cbabb8-9fc6-460b-be61-a59b842e4059 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:40:01.706898 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 583/1162] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:40:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:40:01.712486 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b338a572-f25c-4038-8f98-478a4bca0c3f None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:40:01.712929 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b338a572-f25c-4038-8f98-478a4bca0c3f None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:40:01.713037 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b338a572-f25c-4038-8f98-478a4bca0c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:40:01.731869 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-b338a572-f25c-4038-8f98-478a4bca0c3f None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:40:01.733071 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-b338a572-f25c-4038-8f98-478a4bca0c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:40:01.798826 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 580/1163] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:40:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:40:01.821969 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cbabb8-9fc6-460b-be61-a59b842e4059 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:40:01.884546 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:40:01.924034 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cbabb8-9fc6-460b-be61-a59b842e4059 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:40:01.957212 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5dcd9d-13ef-43e5-b671-af9a19a088cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:40:01.957817 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5dcd9d-13ef-43e5-b671-af9a19a088cf None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:40:01.958633 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5dcd9d-13ef-43e5-b671-af9a19a088cf None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:40:01.958907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5dcd9d-13ef-43e5-b671-af9a19a088cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:40:02.029517 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5dcd9d-13ef-43e5-b671-af9a19a088cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:40:02.030974 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5dcd9d-13ef-43e5-b671-af9a19a088cf None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:40:02.089526 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:40:02.127597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 584/1164] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:40:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 12:40:02.134187 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b338a572-f25c-4038-8f98-478a4bca0c3f None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:40:02.185839 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:40:02.232513 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b338a572-f25c-4038-8f98-478a4bca0c3f None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:40:02.267785 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8579e7-828e-44b9-a02b-3975f8bd1577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:40:02.268421 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8579e7-828e-44b9-a02b-3975f8bd1577 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:40:02.268704 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8579e7-828e-44b9-a02b-3975f8bd1577 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:40:02.268970 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8579e7-828e-44b9-a02b-3975f8bd1577 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:40:02.276252 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8579e7-828e-44b9-a02b-3975f8bd1577 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:40:02.277701 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8579e7-828e-44b9-a02b-3975f8bd1577 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:40:02.288983 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-df8579e7-828e-44b9-a02b-3975f8bd1577 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 26 12:40:02.288983 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:40:02.288983 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 26 12:40:02.288983 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 26 12:40:02.288983 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 12:40:02.288983 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 12:40:02.288983 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 12:40:02.288983 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 12:40:02.288983 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 26 12:40:02.288983 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:40:02.288983 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 12:40:02.288983 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:40:02.288983 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:40:02.288983 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:40:02.288983 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:40:02.288983 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:40:02.288983 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:40:02.290036 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:40:02.290036 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:40:02.290036 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:40:02.290036 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:40:02.290036 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:40:02.290036 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:40:02.290036 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:40:02.290036 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:40:02.290036 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:40:02.290036 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:40:02.290036 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:40:02.290036 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:40:02.290036 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:40:02.290036 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:40:02.290036 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:40:02.290036 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:40:02.290966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:40:02.290966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:40:02.290966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:40:02.290966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:40:02.290966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:40:02.290966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:40:02.290966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:40:02.290966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:40:02.290966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:40:02.290966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:40:02.290966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 26 12:40:02.290966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:40:02.290966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 26 12:40:02.290966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:40:02.290966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 581/1165] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Jul 26 12:40:02 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:40:02.296224 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5dcd9d-13ef-43e5-b671-af9a19a088cf None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:40:02.393381 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5dcd9d-13ef-43e5-b671-af9a19a088cf None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:40:02.426345 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a17901-ecd8-409b-92ff-63682c99cc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:40:02.426936 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a17901-ecd8-409b-92ff-63682c99cc67 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:40:02.427211 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a17901-ecd8-409b-92ff-63682c99cc67 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:40:02.427506 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a17901-ecd8-409b-92ff-63682c99cc67 None admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:40:02.428219 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a17901-ecd8-409b-92ff-63682c99cc67 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:40:02.428513 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a17901-ecd8-409b-92ff-63682c99cc67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:40:02.429895 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a17901-ecd8-409b-92ff-63682c99cc67 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:40:02.437184 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 585/1166] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Jul 26 12:40:02 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:40:04.417460 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099b288-aa2f-4fc4-ba84-08a332f49de7 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:40:04.418000 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099b288-aa2f-4fc4-ba84-08a332f49de7 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:40:04.418000 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099b288-aa2f-4fc4-ba84-08a332f49de7 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:40:04.419365 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 582/1167] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:40:04.424704 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015f0de-8fcc-47d4-a097-49ce45546296 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:40:04.425083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015f0de-8fcc-47d4-a097-49ce45546296 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:40:04.425307 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015f0de-8fcc-47d4-a097-49ce45546296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:40:04.444128 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-6015f0de-8fcc-47d4-a097-49ce45546296 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:40:04.445247 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-6015f0de-8fcc-47d4-a097-49ce45546296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:40:04.467129 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:40:04.511194 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 586/1168] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:40:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:40:04.533680 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d099b288-aa2f-4fc4-ba84-08a332f49de7 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:40:04.635838 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d099b288-aa2f-4fc4-ba84-08a332f49de7 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:40:04.648990 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:40:04.670307 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec45f4e-536a-44bb-8120-945e0dfd30c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:40:04.670915 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec45f4e-536a-44bb-8120-945e0dfd30c2 None admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:40:04.671199 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec45f4e-536a-44bb-8120-945e0dfd30c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:40:04.671474 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec45f4e-536a-44bb-8120-945e0dfd30c2 None admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:40:04.672217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec45f4e-536a-44bb-8120-945e0dfd30c2 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:40:04.673578 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec45f4e-536a-44bb-8120-945e0dfd30c2 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:40:04.688757 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 583/1169] 10.222.0.20 () {60 vars in 1138 bytes} [Tue Jul 26 12:40:04 2022] POST /identity/v3/domains => generated 267 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:40:06.719091 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e36c64d-b34e-4a3e-adc8-871429294474 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:40:06.719661 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e36c64d-b34e-4a3e-adc8-871429294474 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:40:06.719661 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e36c64d-b34e-4a3e-adc8-871429294474 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:40:06.721017 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 587/1170] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:40:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:40:06.726693 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40a44b-c019-4083-870b-fc91c6fc5d9a None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:40:06.727114 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40a44b-c019-4083-870b-fc91c6fc5d9a None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:40:06.727245 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40a44b-c019-4083-870b-fc91c6fc5d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:40:06.745798 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-ff40a44b-c019-4083-870b-fc91c6fc5d9a None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:40:06.747002 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-ff40a44b-c019-4083-870b-fc91c6fc5d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:40:06.811886 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 584/1171] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:40:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:40:06.836542 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e36c64d-b34e-4a3e-adc8-871429294474 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:40:06.939011 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e36c64d-b34e-4a3e-adc8-871429294474 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:40:06.972021 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6db92c-1070-4dc8-ab2d-bd0cd9628ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:40:06.972590 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6db92c-1070-4dc8-ab2d-bd0cd9628ee3 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:40:06.972868 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6db92c-1070-4dc8-ab2d-bd0cd9628ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:40:06.973144 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6db92c-1070-4dc8-ab2d-bd0cd9628ee3 None admin] PATH_INFO: `/v3/domains/d3ddc932162545e9933cfdbbec2ecff9` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:40:06.983816 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6db92c-1070-4dc8-ab2d-bd0cd9628ee3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d3ddc932162545e9933cfdbbec2ecff9)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:40:06.985268 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6db92c-1070-4dc8-ab2d-bd0cd9628ee3 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:40:06.986862 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 588/1172] 10.222.0.20 () {56 vars in 1183 bytes} [Tue Jul 26 12:40:06 2022] GET /identity/v3/domains/d3ddc932162545e9933cfdbbec2ecff9 => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:40:06.993164 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40a44b-c019-4083-870b-fc91c6fc5d9a None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:40:07.092510 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40a44b-c019-4083-870b-fc91c6fc5d9a None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:40:07.094061 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:40:07.128769 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74189c8d-ab75-4d58-b8d4-f328e4e6c86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:40:07.129380 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74189c8d-ab75-4d58-b8d4-f328e4e6c86c None admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:40:07.129659 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74189c8d-ab75-4d58-b8d4-f328e4e6c86c None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:40:07.130502 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74189c8d-ab75-4d58-b8d4-f328e4e6c86c None admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:40:07.131183 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74189c8d-ab75-4d58-b8d4-f328e4e6c86c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:40:07.132646 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74189c8d-ab75-4d58-b8d4-f328e4e6c86c None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:40:07.158025 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 585/1173] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Jul 26 12:40:06 2022] POST /identity/v3/projects => generated 361 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:40:09.185692 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a6df19-096e-4ab4-9210-d296c1e89364 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:40:09.186173 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a6df19-096e-4ab4-9210-d296c1e89364 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:40:09.186284 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a6df19-096e-4ab4-9210-d296c1e89364 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:40:09.187591 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 589/1174] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:40:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:40:09.193558 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ae470-6b77-4484-8a63-213f8f9f5430 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:40:09.193960 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ae470-6b77-4484-8a63-213f8f9f5430 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:40:09.194108 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ae470-6b77-4484-8a63-213f8f9f5430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:40:09.211994 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-fe8ae470-6b77-4484-8a63-213f8f9f5430 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:40:09.213177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-fe8ae470-6b77-4484-8a63-213f8f9f5430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:40:09.278937 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 586/1175] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:40:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:40:09.303356 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a6df19-096e-4ab4-9210-d296c1e89364 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:40:09.323070 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:40:09.406570 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a6df19-096e-4ab4-9210-d296c1e89364 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:40:09.439094 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a6419f-1f27-41a7-a821-c772e6acc2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:40:09.439687 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6419f-1f27-41a7-a821-c772e6acc2b2 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:40:09.439972 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6419f-1f27-41a7-a821-c772e6acc2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:40:09.440248 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6419f-1f27-41a7-a821-c772e6acc2b2 None admin] PATH_INFO: `/v3/domains/d3ddc932162545e9933cfdbbec2ecff9` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:40:09.441401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a6419f-1f27-41a7-a821-c772e6acc2b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d3ddc932162545e9933cfdbbec2ecff9)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:40:09.443397 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a6419f-1f27-41a7-a821-c772e6acc2b2 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:40:09.445010 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 590/1176] 10.222.0.20 () {56 vars in 1183 bytes} [Tue Jul 26 12:40:09 2022] GET /identity/v3/domains/d3ddc932162545e9933cfdbbec2ecff9 => generated 267 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:40:09.450891 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ae470-6b77-4484-8a63-213f8f9f5430 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:40:09.531361 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:40:09.548680 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ae470-6b77-4484-8a63-213f8f9f5430 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:40:09.583360 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da326c30-039d-47cd-b071-18512458ac37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:40:09.583997 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da326c30-039d-47cd-b071-18512458ac37 None admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:40:09.584279 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da326c30-039d-47cd-b071-18512458ac37 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:40:09.584554 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da326c30-039d-47cd-b071-18512458ac37 None admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:40:09.585240 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da326c30-039d-47cd-b071-18512458ac37 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:40:09.586676 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da326c30-039d-47cd-b071-18512458ac37 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:40:09.607795 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 587/1177] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Jul 26 12:40:09 2022] POST /identity/v3/users => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:40:11.646031 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc6a2a-998f-4994-9372-423068d7f486 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:40:11.646594 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc6a2a-998f-4994-9372-423068d7f486 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:40:11.646594 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc6a2a-998f-4994-9372-423068d7f486 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:40:11.647932 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 591/1178] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:40:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:40:11.653195 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28946a1a-218b-4086-bbc6-cf29db774c0d None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:40:11.653589 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28946a1a-218b-4086-bbc6-cf29db774c0d None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:40:11.653748 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28946a1a-218b-4086-bbc6-cf29db774c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:40:11.672100 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-28946a1a-218b-4086-bbc6-cf29db774c0d None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:40:11.673293 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-28946a1a-218b-4086-bbc6-cf29db774c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:40:11.741383 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 588/1179] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:40:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:40:11.765117 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cc6a2a-998f-4994-9372-423068d7f486 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:40:11.856026 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:40:11.873728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cc6a2a-998f-4994-9372-423068d7f486 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:40:11.910861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86051c73-e83f-4d59-ac68-0f2a760a1ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:40:11.911436 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86051c73-e83f-4d59-ac68-0f2a760a1ae4 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:40:11.911757 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86051c73-e83f-4d59-ac68-0f2a760a1ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:40:11.912035 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86051c73-e83f-4d59-ac68-0f2a760a1ae4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:40:11.923947 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86051c73-e83f-4d59-ac68-0f2a760a1ae4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:40:11.928993 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-86051c73-e83f-4d59-ac68-0f2a760a1ae4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:40:11.928993 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:40:11.928993 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:40:11.928993 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:40:11.928993 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:40:11.928993 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:40:11.928993 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:40:11.928993 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:40:11.928993 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:40:11.928993 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:40:11.928993 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:40:11.928993 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:40:11.928993 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:40:11.928993 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:40:11.928993 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:40:11.928993 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:40:11.930245 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:40:11.930245 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:40:11.930245 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:40:11.930245 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:40:11.930245 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:40:11.930245 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:40:11.930245 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:40:11.930245 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:40:11.930245 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:40:11.930245 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:40:11.930245 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:40:11.930245 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:40:11.930245 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:40:11.930245 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:40:11.930245 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:40:11.930245 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:40:11.931335 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:40:11.931335 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:40:11.931335 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:40:11.931335 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:40:11.931335 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:40:11.931335 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:40:11.931335 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:40:11.931335 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:40:11.931335 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:40:11.931335 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:40:11.931335 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:40:11.931335 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:40:11.931335 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 592/1180] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 26 12:40:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:40:11.936420 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28946a1a-218b-4086-bbc6-cf29db774c0d None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:40:12.002481 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:40:12.038174 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28946a1a-218b-4086-bbc6-cf29db774c0d None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:40:12.070951 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db1a8b6-b544-4d83-89f7-2e33e1a59741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:40:12.071553 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1a8b6-b544-4d83-89f7-2e33e1a59741 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:40:12.071859 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1a8b6-b544-4d83-89f7-2e33e1a59741 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:40:12.072130 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1a8b6-b544-4d83-89f7-2e33e1a59741 None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:40:12.072840 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db1a8b6-b544-4d83-89f7-2e33e1a59741 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:40:12.073137 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db1a8b6-b544-4d83-89f7-2e33e1a59741 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:40:12.074499 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db1a8b6-b544-4d83-89f7-2e33e1a59741 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:40:12.082821 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 589/1181] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:40:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:40:12.089190 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86051c73-e83f-4d59-ac68-0f2a760a1ae4 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:40:12.195307 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86051c73-e83f-4d59-ac68-0f2a760a1ae4 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:40:12.227788 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a2030b-886c-4ea7-99ad-d4ec625a3b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:40:12.228416 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a2030b-886c-4ea7-99ad-d4ec625a3b2b None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:40:12.228700 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a2030b-886c-4ea7-99ad-d4ec625a3b2b None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:40:12.228977 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a2030b-886c-4ea7-99ad-d4ec625a3b2b None admin] PATH_INFO: `/v3/users/38639ada2f2f434c8eea9f68f3a3bd02` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:40:12.243004 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a2030b-886c-4ea7-99ad-d4ec625a3b2b None admin] RBAC: Authorizing `identity:get_user(user_id=38639ada2f2f434c8eea9f68f3a3bd02)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:40:12.244484 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a2030b-886c-4ea7-99ad-d4ec625a3b2b None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:40:12.246112 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 593/1182] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:40:12 2022] GET /identity/v3/users/38639ada2f2f434c8eea9f68f3a3bd02 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:40:12.252132 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db1a8b6-b544-4d83-89f7-2e33e1a59741 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:40:12.355981 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db1a8b6-b544-4d83-89f7-2e33e1a59741 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:40:12.402328 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:40:12.408063 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6cbd34-d459-4a2a-8621-b232f008ec4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:40:12.408687 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6cbd34-d459-4a2a-8621-b232f008ec4a None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:40:12.408967 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6cbd34-d459-4a2a-8621-b232f008ec4a None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:40:12.409235 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6cbd34-d459-4a2a-8621-b232f008ec4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:40:12.478346 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6cbd34-d459-4a2a-8621-b232f008ec4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:40:12.479975 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6cbd34-d459-4a2a-8621-b232f008ec4a None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:40:12.575874 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 590/1183] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:40:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 26 12:40:12.581761 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a2030b-886c-4ea7-99ad-d4ec625a3b2b None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:40:12.586069 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:40:12.680810 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a2030b-886c-4ea7-99ad-d4ec625a3b2b None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:40:12.714289 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf65bdc-d0f9-487d-8563-d250c5dc47b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:40:12.714928 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf65bdc-d0f9-487d-8563-d250c5dc47b6 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:40:12.715207 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf65bdc-d0f9-487d-8563-d250c5dc47b6 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:40:12.715480 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf65bdc-d0f9-487d-8563-d250c5dc47b6 None admin] PATH_INFO: `/v3/projects/7f1b21e0c6004c6387a928497434124b` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:40:12.716683 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf65bdc-d0f9-487d-8563-d250c5dc47b6 None admin] RBAC: Authorizing `identity:get_project(project_id=7f1b21e0c6004c6387a928497434124b)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:40:12.718159 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf65bdc-d0f9-487d-8563-d250c5dc47b6 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:40:12.719933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 594/1184] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:40:12 2022] GET /identity/v3/projects/7f1b21e0c6004c6387a928497434124b => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:40:12.726311 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6cbd34-d459-4a2a-8621-b232f008ec4a None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:40:12.759589 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:40:12.828607 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6cbd34-d459-4a2a-8621-b232f008ec4a None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:40:12.862294 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4ea44-39e6-4515-9d76-a8ae92db12d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:40:12.862904 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4ea44-39e6-4515-9d76-a8ae92db12d3 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:40:12.863197 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4ea44-39e6-4515-9d76-a8ae92db12d3 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:40:12.863456 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4ea44-39e6-4515-9d76-a8ae92db12d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:40:12.864781 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4ea44-39e6-4515-9d76-a8ae92db12d3 None admin] RBAC: Adding query filter params (role.id=0e78444e43c744668178971af4087ca0, scope.project.id=7f1b21e0c6004c6387a928497434124b, user.id=38639ada2f2f434c8eea9f68f3a3bd02) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:40:12.865093 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4ea44-39e6-4515-9d76-a8ae92db12d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:40:12.866779 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4ea44-39e6-4515-9d76-a8ae92db12d3 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:40:12.872127 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 591/1185] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:40:12 2022] GET /identity/v3/role_assignments?user.id=38639ada2f2f434c8eea9f68f3a3bd02&scope.project.id=7f1b21e0c6004c6387a928497434124b&role.id=0e78444e43c744668178971af4087ca0 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:40:14.845809 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40b6e8-f62d-4843-804c-b4cd2be6ee75 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:40:14.846367 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40b6e8-f62d-4843-804c-b4cd2be6ee75 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:40:14.846367 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40b6e8-f62d-4843-804c-b4cd2be6ee75 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:40:14.847699 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 595/1186] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:40:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:40:14.853251 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b1c4c6-6e58-4227-8242-dcfcac98111d None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:40:14.853633 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b1c4c6-6e58-4227-8242-dcfcac98111d None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:40:14.853808 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b1c4c6-6e58-4227-8242-dcfcac98111d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:40:14.872996 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-16b1c4c6-6e58-4227-8242-dcfcac98111d None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:40:14.874199 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-16b1c4c6-6e58-4227-8242-dcfcac98111d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:40:14.939497 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 592/1187] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:40:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:40:14.961242 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a40b6e8-f62d-4843-804c-b4cd2be6ee75 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:40:15.032500 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:40:15.063773 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a40b6e8-f62d-4843-804c-b4cd2be6ee75 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:40:15.096135 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb6647-6db9-4e02-9951-ba49668847df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:40:15.096686 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb6647-6db9-4e02-9951-ba49668847df None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:40:15.096963 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb6647-6db9-4e02-9951-ba49668847df None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:40:15.097229 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb6647-6db9-4e02-9951-ba49668847df None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:40:15.107519 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbb6647-6db9-4e02-9951-ba49668847df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:40:15.112346 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-8bbb6647-6db9-4e02-9951-ba49668847df None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:40:15.112346 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:40:15.112346 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:40:15.112346 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:40:15.112346 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:40:15.112346 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:40:15.112346 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:40:15.112346 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:40:15.112346 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:40:15.112346 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:40:15.112346 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:40:15.112346 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:40:15.112346 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:40:15.112346 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:40:15.112346 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:40:15.112346 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:40:15.113323 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:40:15.113323 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:40:15.113323 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:40:15.113323 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:40:15.113323 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:40:15.113323 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:40:15.113323 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:40:15.113323 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:40:15.113323 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:40:15.113323 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:40:15.113323 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:40:15.113323 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:40:15.113323 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:40:15.113323 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:40:15.113323 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:40:15.113323 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:40:15.114408 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:40:15.114408 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:40:15.114408 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:40:15.114408 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:40:15.114408 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:40:15.114408 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:40:15.114408 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:40:15.114408 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:40:15.114408 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:40:15.114408 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:40:15.114408 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:40:15.114408 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:40:15.114408 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 596/1188] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 26 12:40:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:40:15.119812 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b1c4c6-6e58-4227-8242-dcfcac98111d None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:40:15.146432 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:40:15.217162 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b1c4c6-6e58-4227-8242-dcfcac98111d None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:40:15.250459 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d03f794-2a0a-41c0-939e-bc53ba33c133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:40:15.251071 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d03f794-2a0a-41c0-939e-bc53ba33c133 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:40:15.251355 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d03f794-2a0a-41c0-939e-bc53ba33c133 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:40:15.251630 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d03f794-2a0a-41c0-939e-bc53ba33c133 None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:40:15.252364 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d03f794-2a0a-41c0-939e-bc53ba33c133 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:40:15.252670 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d03f794-2a0a-41c0-939e-bc53ba33c133 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:40:15.254055 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d03f794-2a0a-41c0-939e-bc53ba33c133 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:40:15.261936 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 593/1189] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:40:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:40:15.267822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb6647-6db9-4e02-9951-ba49668847df None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:40:15.369520 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb6647-6db9-4e02-9951-ba49668847df None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:40:15.398457 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:40:15.403964 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16134f62-54dd-41c5-9599-117a7b2e3b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:40:15.404605 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16134f62-54dd-41c5-9599-117a7b2e3b03 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:40:15.404889 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16134f62-54dd-41c5-9599-117a7b2e3b03 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:40:15.405165 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16134f62-54dd-41c5-9599-117a7b2e3b03 None admin] PATH_INFO: `/v3/users/38639ada2f2f434c8eea9f68f3a3bd02` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:40:15.406326 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16134f62-54dd-41c5-9599-117a7b2e3b03 None admin] RBAC: Authorizing `identity:get_user(user_id=38639ada2f2f434c8eea9f68f3a3bd02)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:40:15.407795 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16134f62-54dd-41c5-9599-117a7b2e3b03 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:40:15.409409 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 597/1190] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:40:15 2022] GET /identity/v3/users/38639ada2f2f434c8eea9f68f3a3bd02 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:40:15.415849 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d03f794-2a0a-41c0-939e-bc53ba33c133 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:40:15.500035 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:40:15.512455 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d03f794-2a0a-41c0-939e-bc53ba33c133 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:40:15.544473 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb8997-2a53-48b8-9ef7-1a2ec49db255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:40:15.545087 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb8997-2a53-48b8-9ef7-1a2ec49db255 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:40:15.545368 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb8997-2a53-48b8-9ef7-1a2ec49db255 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:40:15.545636 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb8997-2a53-48b8-9ef7-1a2ec49db255 None admin] PATH_INFO: `/v3/projects/7f1b21e0c6004c6387a928497434124b` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:40:15.546768 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb8997-2a53-48b8-9ef7-1a2ec49db255 None admin] RBAC: Authorizing `identity:get_project(project_id=7f1b21e0c6004c6387a928497434124b)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:40:15.548225 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb8997-2a53-48b8-9ef7-1a2ec49db255 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:40:15.549865 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 594/1191] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:40:15 2022] GET /identity/v3/projects/7f1b21e0c6004c6387a928497434124b => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:40:15.555848 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16134f62-54dd-41c5-9599-117a7b2e3b03 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:40:15.656287 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16134f62-54dd-41c5-9599-117a7b2e3b03 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:40:15.689558 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99fa49a-4cee-40ee-a054-4c4478cc5f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:40:15.690236 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99fa49a-4cee-40ee-a054-4c4478cc5f89 None admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:40:15.690519 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99fa49a-4cee-40ee-a054-4c4478cc5f89 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:40:15.690799 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99fa49a-4cee-40ee-a054-4c4478cc5f89 None admin] PATH_INFO: `/v3/projects/7f1b21e0c6004c6387a928497434124b/users/38639ada2f2f434c8eea9f68f3a3bd02/roles/0e78444e43c744668178971af4087ca0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:40:15.693495 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99fa49a-4cee-40ee-a054-4c4478cc5f89 None admin] RBAC: Authorizing `identity:create_grant(project_id=7f1b21e0c6004c6387a928497434124b, user_id=38639ada2f2f434c8eea9f68f3a3bd02, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:40:15.694949 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99fa49a-4cee-40ee-a054-4c4478cc5f89 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:40:15.713316 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 598/1192] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 26 12:40:15 2022] PUT /identity/v3/projects/7f1b21e0c6004c6387a928497434124b/users/38639ada2f2f434c8eea9f68f3a3bd02/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:40:17.693497 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77280bda-2a99-44ee-a58b-0266f340f44e None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:40:17.694058 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77280bda-2a99-44ee-a58b-0266f340f44e None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:40:17.694058 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77280bda-2a99-44ee-a58b-0266f340f44e None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:40:17.695388 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 595/1193] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:40:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:40:17.700573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8feca8-6961-49a4-bc8e-8be436fc1aa6 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:40:17.700954 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8feca8-6961-49a4-bc8e-8be436fc1aa6 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:40:17.701123 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8feca8-6961-49a4-bc8e-8be436fc1aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:40:17.719252 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-ae8feca8-6961-49a4-bc8e-8be436fc1aa6 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:40:17.720477 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-ae8feca8-6961-49a4-bc8e-8be436fc1aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:40:17.775942 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:40:17.788271 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 599/1194] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:40:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 12:40:17.811470 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77280bda-2a99-44ee-a58b-0266f340f44e None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:40:17.912886 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77280bda-2a99-44ee-a58b-0266f340f44e None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:40:17.946064 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eb8418-f6d6-4ccf-bafd-618eab7f3d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:40:17.946630 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eb8418-f6d6-4ccf-bafd-618eab7f3d82 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:40:17.946912 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eb8418-f6d6-4ccf-bafd-618eab7f3d82 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:40:17.947187 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eb8418-f6d6-4ccf-bafd-618eab7f3d82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:40:17.957134 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eb8418-f6d6-4ccf-bafd-618eab7f3d82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:40:17.962074 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-66eb8418-f6d6-4ccf-bafd-618eab7f3d82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:40:17.962074 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:40:17.962074 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:40:17.962074 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:40:17.962074 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:40:17.962074 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:40:17.962074 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:40:17.962074 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:40:17.962074 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:40:17.962074 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:40:17.962074 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:40:17.962074 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:40:17.962074 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:40:17.962074 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:40:17.962074 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:40:17.962074 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:40:17.963155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:40:17.963155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:40:17.963155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:40:17.963155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:40:17.963155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:40:17.963155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:40:17.963155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:40:17.963155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:40:17.963155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:40:17.963155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:40:17.963155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:40:17.963155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:40:17.963155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:40:17.963155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:40:17.963155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:40:17.963155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:40:17.964099 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:40:17.964099 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:40:17.964099 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:40:17.964099 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:40:17.964099 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:40:17.964099 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:40:17.964099 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:40:17.964099 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:40:17.964099 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:40:17.964099 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:40:17.964099 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:40:17.964099 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:40:17.964099 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 596/1195] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 26 12:40:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:40:17.969551 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8feca8-6961-49a4-bc8e-8be436fc1aa6 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:40:18.066281 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8feca8-6961-49a4-bc8e-8be436fc1aa6 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:40:18.100141 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d9f9df-09a2-42c2-9ca3-9624261a6294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:40:18.100740 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9f9df-09a2-42c2-9ca3-9624261a6294 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:40:18.101024 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9f9df-09a2-42c2-9ca3-9624261a6294 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:40:18.101299 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9f9df-09a2-42c2-9ca3-9624261a6294 None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:40:18.102008 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d9f9df-09a2-42c2-9ca3-9624261a6294 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:40:18.102315 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d9f9df-09a2-42c2-9ca3-9624261a6294 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:40:18.103716 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d9f9df-09a2-42c2-9ca3-9624261a6294 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:40:18.111552 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 600/1196] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:40:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:40:18.117608 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eb8418-f6d6-4ccf-bafd-618eab7f3d82 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:40:18.135986 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:40:18.219924 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eb8418-f6d6-4ccf-bafd-618eab7f3d82 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:40:18.253145 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a06d2d-15e5-4544-8739-2c60fcd0b026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:40:18.253772 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a06d2d-15e5-4544-8739-2c60fcd0b026 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:40:18.254053 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a06d2d-15e5-4544-8739-2c60fcd0b026 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:40:18.254325 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a06d2d-15e5-4544-8739-2c60fcd0b026 None admin] PATH_INFO: `/v3/users/38639ada2f2f434c8eea9f68f3a3bd02` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:40:18.255467 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a06d2d-15e5-4544-8739-2c60fcd0b026 None admin] RBAC: Authorizing `identity:get_user(user_id=38639ada2f2f434c8eea9f68f3a3bd02)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:40:18.256926 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a06d2d-15e5-4544-8739-2c60fcd0b026 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:40:18.258527 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 597/1197] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 26 12:40:18 2022] GET /identity/v3/users/38639ada2f2f434c8eea9f68f3a3bd02 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:40:18.264373 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d9f9df-09a2-42c2-9ca3-9624261a6294 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:40:18.316573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:40:18.365501 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d9f9df-09a2-42c2-9ca3-9624261a6294 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:40:18.398397 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352bd67e-583e-4d54-b2dc-44f025a9b949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:40:18.399004 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352bd67e-583e-4d54-b2dc-44f025a9b949 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:40:18.399284 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352bd67e-583e-4d54-b2dc-44f025a9b949 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:40:18.399556 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352bd67e-583e-4d54-b2dc-44f025a9b949 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:40:18.465030 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352bd67e-583e-4d54-b2dc-44f025a9b949 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:40:18.466478 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352bd67e-583e-4d54-b2dc-44f025a9b949 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:40:18.515995 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:40:18.562237 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 601/1198] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:40:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 12:40:18.568633 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a06d2d-15e5-4544-8739-2c60fcd0b026 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:40:18.650628 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:40:18.666335 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a06d2d-15e5-4544-8739-2c60fcd0b026 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:40:18.699422 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee97b542-2b9e-4352-81ff-617465dc3e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:40:18.700069 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97b542-2b9e-4352-81ff-617465dc3e48 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:40:18.700351 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97b542-2b9e-4352-81ff-617465dc3e48 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:40:18.700620 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97b542-2b9e-4352-81ff-617465dc3e48 None admin] PATH_INFO: `/v3/projects/7f1b21e0c6004c6387a928497434124b` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:40:18.701759 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee97b542-2b9e-4352-81ff-617465dc3e48 None admin] RBAC: Authorizing `identity:get_project(project_id=7f1b21e0c6004c6387a928497434124b)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:40:18.703179 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee97b542-2b9e-4352-81ff-617465dc3e48 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:40:18.704863 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 598/1199] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 26 12:40:18 2022] GET /identity/v3/projects/7f1b21e0c6004c6387a928497434124b => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:40:18.711733 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352bd67e-583e-4d54-b2dc-44f025a9b949 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:40:18.811148 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352bd67e-583e-4d54-b2dc-44f025a9b949 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:40:18.845014 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb40e0b-39fc-4ec5-87f8-381bb8b68d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:40:18.845615 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb40e0b-39fc-4ec5-87f8-381bb8b68d5f None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:40:18.845894 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb40e0b-39fc-4ec5-87f8-381bb8b68d5f None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:40:18.846166 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb40e0b-39fc-4ec5-87f8-381bb8b68d5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:40:18.846945 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb40e0b-39fc-4ec5-87f8-381bb8b68d5f None admin] RBAC: Adding query filter params (role.id=0e78444e43c744668178971af4087ca0, scope.project.id=7f1b21e0c6004c6387a928497434124b, user.id=38639ada2f2f434c8eea9f68f3a3bd02) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:40:18.847252 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb40e0b-39fc-4ec5-87f8-381bb8b68d5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:40:18.848747 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb40e0b-39fc-4ec5-87f8-381bb8b68d5f None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:40:18.854398 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 602/1200] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:40:18 2022] GET /identity/v3/role_assignments?user.id=38639ada2f2f434c8eea9f68f3a3bd02&scope.project.id=7f1b21e0c6004c6387a928497434124b&role.id=0e78444e43c744668178971af4087ca0 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:40:28.695245 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=24 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:40:28.851835 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=92 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:14.195420 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ffb08-7cde-402b-bf12-e47ce0843854 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:14.196020 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ffb08-7cde-402b-bf12-e47ce0843854 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:14.196020 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ffb08-7cde-402b-bf12-e47ce0843854 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:14.197351 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 599/1201] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:41:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:41:14.203761 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c74198-8200-47f2-ab34-d56ca8a7cf81 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:14.204295 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c74198-8200-47f2-ab34-d56ca8a7cf81 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:14.204295 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c74198-8200-47f2-ab34-d56ca8a7cf81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:14.230622 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-01c74198-8200-47f2-ab34-d56ca8a7cf81 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:41:14.231982 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-01c74198-8200-47f2-ab34-d56ca8a7cf81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:41:14.322827 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 603/1202] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:41:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 12:41:14.348389 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ffb08-7cde-402b-bf12-e47ce0843854 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:14.510274 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ffb08-7cde-402b-bf12-e47ce0843854 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:14.552445 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b20f2a4-e94b-464a-a63b-74a4e7bf25d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:14.553522 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20f2a4-e94b-464a-a63b-74a4e7bf25d0 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:14.554046 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20f2a4-e94b-464a-a63b-74a4e7bf25d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:14.554555 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20f2a4-e94b-464a-a63b-74a4e7bf25d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:14.568485 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b20f2a4-e94b-464a-a63b-74a4e7bf25d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:14.570123 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b20f2a4-e94b-464a-a63b-74a4e7bf25d0 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:14.592361 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-9b20f2a4-e94b-464a-a63b-74a4e7bf25d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:41:14.592361 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:41:14.592361 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:41:14.592361 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:41:14.592361 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:41:14.592361 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:41:14.592361 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:41:14.592361 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:41:14.592361 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:41:14.592361 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:41:14.592361 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:41:14.592361 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:41:14.592361 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:41:14.592361 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:41:14.592361 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:41:14.593717 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:41:14.593717 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:41:14.593717 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:41:14.593717 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:41:14.593717 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:41:14.593717 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:41:14.593717 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:41:14.593717 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:41:14.593717 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:41:14.593717 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:41:14.593717 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:41:14.593717 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:41:14.593717 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:41:14.593717 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:41:14.593717 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:41:14.593717 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:41:14.593717 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:41:14.595174 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:41:14.595174 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:41:14.595174 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:41:14.595174 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:41:14.595174 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 600/1203] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:41:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:41:14.599882 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c74198-8200-47f2-ab34-d56ca8a7cf81 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:14.746858 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c74198-8200-47f2-ab34-d56ca8a7cf81 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:14.789663 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:14.797371 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19976924-9545-46d0-b3c2-410bbfab3a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:14.798478 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19976924-9545-46d0-b3c2-410bbfab3a59 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:14.799063 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19976924-9545-46d0-b3c2-410bbfab3a59 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:14.799807 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19976924-9545-46d0-b3c2-410bbfab3a59 None admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:14.800699 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19976924-9545-46d0-b3c2-410bbfab3a59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:41:14.801679 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19976924-9545-46d0-b3c2-410bbfab3a59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:14.803444 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19976924-9545-46d0-b3c2-410bbfab3a59 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:14.820617 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 604/1204] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:41:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:41:14.828333 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b20f2a4-e94b-464a-a63b-74a4e7bf25d0 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:14.899999 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:14.975107 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b20f2a4-e94b-464a-a63b-74a4e7bf25d0 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:15.023463 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e5421-5dff-4e29-9702-f95880f3563d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:15.024197 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e5421-5dff-4e29-9702-f95880f3563d None admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:15.024563 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e5421-5dff-4e29-9702-f95880f3563d None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:15.024864 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e5421-5dff-4e29-9702-f95880f3563d None admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:15.025707 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7e5421-5dff-4e29-9702-f95880f3563d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:15.027664 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7e5421-5dff-4e29-9702-f95880f3563d None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:15.054642 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 601/1205] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Jul 26 12:41:14 2022] POST /identity/v3/users => generated 251 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:41:17.434613 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8db6c4-0066-46be-a352-cdfb2b7a72b0 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:17.435163 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8db6c4-0066-46be-a352-cdfb2b7a72b0 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:17.435431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8db6c4-0066-46be-a352-cdfb2b7a72b0 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:17.436964 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 605/1206] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:41:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:41:17.443347 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8291f047-314e-4de6-9ffa-9d33008df81d None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:17.443880 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8291f047-314e-4de6-9ffa-9d33008df81d None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:17.443992 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8291f047-314e-4de6-9ffa-9d33008df81d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:17.474315 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-8291f047-314e-4de6-9ffa-9d33008df81d None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:41:17.476037 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-8291f047-314e-4de6-9ffa-9d33008df81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:41:17.560906 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 602/1207] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:41:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:41:17.585428 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8db6c4-0066-46be-a352-cdfb2b7a72b0 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:17.716868 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8db6c4-0066-46be-a352-cdfb2b7a72b0 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:17.757372 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7cac6c-7180-4bf5-bb9b-fa366daf3d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:17.758153 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cac6c-7180-4bf5-bb9b-fa366daf3d0f None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:17.758489 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cac6c-7180-4bf5-bb9b-fa366daf3d0f None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:17.758794 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cac6c-7180-4bf5-bb9b-fa366daf3d0f None admin] PATH_INFO: `/v3/roles/service` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:17.773877 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7cac6c-7180-4bf5-bb9b-fa366daf3d0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:41:17.779265 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-7c7cac6c-7180-4bf5-bb9b-fa366daf3d0f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 12:41:17.779265 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:41:17.779265 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:41:17.779265 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:41:17.779265 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:41:17.779265 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:41:17.779265 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:41:17.779265 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:41:17.779265 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:41:17.779265 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:41:17.779265 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:41:17.779265 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:41:17.779265 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:41:17.779265 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:41:17.779265 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:41:17.779265 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:41:17.780816 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:41:17.780816 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:41:17.780816 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:41:17.780816 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:41:17.780816 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:41:17.780816 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:41:17.780816 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:41:17.780816 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:41:17.780816 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:41:17.780816 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:41:17.780816 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:41:17.780816 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:41:17.780816 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:41:17.780816 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:41:17.780816 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:41:17.780816 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:41:17.781838 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:41:17.781838 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:41:17.781838 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:41:17.781838 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:41:17.781838 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:41:17.781838 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:41:17.781838 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:41:17.781838 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:41:17.781838 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:41:17.781838 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:41:17.781838 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 12:41:17.781838 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:41:17.781838 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 606/1208] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 26 12:41:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:41:17.789269 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8291f047-314e-4de6-9ffa-9d33008df81d None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:17.837470 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:17.913233 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8291f047-314e-4de6-9ffa-9d33008df81d None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:17.954948 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0142a450-5386-4d13-bf7a-2d41c7939bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:17.955593 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142a450-5386-4d13-bf7a-2d41c7939bfa None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:17.955936 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142a450-5386-4d13-bf7a-2d41c7939bfa None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:17.956218 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142a450-5386-4d13-bf7a-2d41c7939bfa None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:17.956957 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0142a450-5386-4d13-bf7a-2d41c7939bfa None admin] RBAC: Adding query filter params (name=service) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:41:17.957781 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0142a450-5386-4d13-bf7a-2d41c7939bfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:17.959154 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0142a450-5386-4d13-bf7a-2d41c7939bfa None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:17.969638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 603/1209] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Jul 26 12:41:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:41:17.975689 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7cac6c-7180-4bf5-bb9b-fa366daf3d0f None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:18.046113 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:18.124098 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7cac6c-7180-4bf5-bb9b-fa366daf3d0f None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:18.182299 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c905e4-90de-4bd2-8929-822e96a4d61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:18.183277 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c905e4-90de-4bd2-8929-822e96a4d61b None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:18.183804 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c905e4-90de-4bd2-8929-822e96a4d61b None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:18.184141 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c905e4-90de-4bd2-8929-822e96a4d61b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:18.201806 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c905e4-90de-4bd2-8929-822e96a4d61b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:18.203741 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c905e4-90de-4bd2-8929-822e96a4d61b None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:18.222349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-69c905e4-90de-4bd2-8929-822e96a4d61b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:41:18.222349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:41:18.222349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:41:18.222349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:41:18.222349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:41:18.222349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:41:18.222349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:41:18.222349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:41:18.222349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:41:18.222349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:41:18.222349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:41:18.222349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:41:18.222349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:41:18.222349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:41:18.222349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:41:18.223334 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:41:18.223334 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:41:18.223334 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:41:18.223334 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:41:18.223334 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:41:18.223334 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:41:18.223334 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:41:18.223334 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:41:18.223334 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:41:18.223334 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:41:18.223334 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:41:18.223334 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:41:18.223334 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:41:18.223334 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:41:18.223334 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:41:18.223334 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:41:18.223334 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:41:18.225262 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:41:18.225262 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:41:18.225262 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:41:18.225262 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:41:18.225262 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 607/1210] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:41:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:41:18.230120 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0142a450-5386-4d13-bf7a-2d41c7939bfa None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:18.338367 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0142a450-5386-4d13-bf7a-2d41c7939bfa None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:18.341841 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:18.377782 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab024f9-0eb1-4a55-ae06-9f13cf47f6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:18.378686 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab024f9-0eb1-4a55-ae06-9f13cf47f6b3 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:18.379025 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab024f9-0eb1-4a55-ae06-9f13cf47f6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:18.379298 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab024f9-0eb1-4a55-ae06-9f13cf47f6b3 None admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:18.380154 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab024f9-0eb1-4a55-ae06-9f13cf47f6b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:41:18.380460 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab024f9-0eb1-4a55-ae06-9f13cf47f6b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:18.382000 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab024f9-0eb1-4a55-ae06-9f13cf47f6b3 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:18.400174 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 604/1211] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:41:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:41:18.401509 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c905e4-90de-4bd2-8929-822e96a4d61b None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:18.518939 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c905e4-90de-4bd2-8929-822e96a4d61b None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:18.521411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:18.558624 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e00996-bca6-4e84-b12f-a6965f31afcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:18.559314 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e00996-bca6-4e84-b12f-a6965f31afcd None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:18.559550 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e00996-bca6-4e84-b12f-a6965f31afcd None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:18.559905 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e00996-bca6-4e84-b12f-a6965f31afcd None admin] PATH_INFO: `/v3/users/placement` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:18.568922 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e00996-bca6-4e84-b12f-a6965f31afcd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:18.570489 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e00996-bca6-4e84-b12f-a6965f31afcd None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:18.583273 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-30e00996-bca6-4e84-b12f-a6965f31afcd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 12:41:18.583273 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:41:18.583273 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:41:18.583273 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:41:18.583273 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:41:18.583273 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:41:18.583273 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:41:18.583273 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:41:18.583273 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:41:18.583273 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:41:18.583273 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:41:18.583273 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:41:18.583273 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 12:41:18.583273 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 12:41:18.583273 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 12:41:18.583273 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 12:41:18.584565 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:41:18.584565 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:41:18.584565 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 12:41:18.584565 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:41:18.584565 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 12:41:18.584565 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:41:18.584565 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:41:18.584565 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:41:18.584565 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:41:18.584565 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:41:18.584565 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:41:18.584565 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:41:18.584565 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:41:18.584565 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:41:18.584565 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:41:18.584565 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:41:18.584565 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:41:18.584565 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:41:18.585678 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:41:18.585678 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:41:18.585678 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 12:41:18.585678 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 12:41:18.585678 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 12:41:18.585678 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 12:41:18.585678 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 12:41:18.585678 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 12:41:18.585678 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 12:41:18.585678 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:41:18.585678 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 608/1212] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:41:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:41:18.592719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab024f9-0eb1-4a55-ae06-9f13cf47f6b3 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:18.716786 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab024f9-0eb1-4a55-ae06-9f13cf47f6b3 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:18.757987 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f879d347-ec93-4993-8ec6-bd2a82638506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:18.758717 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879d347-ec93-4993-8ec6-bd2a82638506 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:18.760424 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879d347-ec93-4993-8ec6-bd2a82638506 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:18.760424 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879d347-ec93-4993-8ec6-bd2a82638506 None admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:18.760424 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f879d347-ec93-4993-8ec6-bd2a82638506 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:41:18.760639 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f879d347-ec93-4993-8ec6-bd2a82638506 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:18.762051 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f879d347-ec93-4993-8ec6-bd2a82638506 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:18.781953 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 605/1213] 10.222.0.20 () {56 vars in 1143 bytes} [Tue Jul 26 12:41:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:41:18.787291 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e00996-bca6-4e84-b12f-a6965f31afcd None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:18.916781 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e00996-bca6-4e84-b12f-a6965f31afcd None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:18.958515 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b48488-a5d3-42f0-b090-c3e11957dd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:18.959141 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b48488-a5d3-42f0-b090-c3e11957dd1c None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:18.959416 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b48488-a5d3-42f0-b090-c3e11957dd1c None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:18.959711 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b48488-a5d3-42f0-b090-c3e11957dd1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:18.996474 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:19.038018 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b48488-a5d3-42f0-b090-c3e11957dd1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:19.039338 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b48488-a5d3-42f0-b090-c3e11957dd1c None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:19.151338 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 609/1214] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:41:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 12:41:19.157055 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f879d347-ec93-4993-8ec6-bd2a82638506 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:19.186930 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:19.283358 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f879d347-ec93-4993-8ec6-bd2a82638506 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:19.325210 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a60ac-3ae4-4266-b3b9-bdb72a1285b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:19.325912 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a60ac-3ae4-4266-b3b9-bdb72a1285b4 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:19.326089 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a60ac-3ae4-4266-b3b9-bdb72a1285b4 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:19.326406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a60ac-3ae4-4266-b3b9-bdb72a1285b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:19.338676 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9a60ac-3ae4-4266-b3b9-bdb72a1285b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:19.340276 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9a60ac-3ae4-4266-b3b9-bdb72a1285b4 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:19.356006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-dc9a60ac-3ae4-4266-b3b9-bdb72a1285b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:41:19.356006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:41:19.356006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:41:19.356006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:41:19.356006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:41:19.356006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:41:19.356006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:41:19.356006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:41:19.356006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:41:19.356006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:41:19.356006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:41:19.356006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:41:19.356006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:41:19.356006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:41:19.356006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:41:19.357314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:41:19.357314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:41:19.357314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:41:19.357314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:41:19.357314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:41:19.357314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:41:19.357314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:41:19.357314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:41:19.357314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:41:19.357314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:41:19.357314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:41:19.357314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:41:19.357314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:41:19.357314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:41:19.357314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:41:19.357314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:41:19.357314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:41:19.358263 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:41:19.358263 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:41:19.358263 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:41:19.358263 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:41:19.358263 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 606/1215] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:41:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:41:19.363060 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b48488-a5d3-42f0-b090-c3e11957dd1c None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:19.443478 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:19.485786 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b48488-a5d3-42f0-b090-c3e11957dd1c None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:19.525455 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d4f362-ef42-4532-8dd6-87ca36636651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:19.526198 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4f362-ef42-4532-8dd6-87ca36636651 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:19.526458 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4f362-ef42-4532-8dd6-87ca36636651 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:19.526665 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4f362-ef42-4532-8dd6-87ca36636651 None admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:19.527412 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d4f362-ef42-4532-8dd6-87ca36636651 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:41:19.527786 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d4f362-ef42-4532-8dd6-87ca36636651 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:19.529302 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d4f362-ef42-4532-8dd6-87ca36636651 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:19.542489 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 610/1216] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:41:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:41:19.548692 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a60ac-3ae4-4266-b3b9-bdb72a1285b4 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:19.635889 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:19.674061 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a60ac-3ae4-4266-b3b9-bdb72a1285b4 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:19.714593 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06acffdf-d798-44d4-82a9-5640e73bc8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:19.715276 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06acffdf-d798-44d4-82a9-5640e73bc8a9 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:19.715582 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06acffdf-d798-44d4-82a9-5640e73bc8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:19.715909 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06acffdf-d798-44d4-82a9-5640e73bc8a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:19.724153 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06acffdf-d798-44d4-82a9-5640e73bc8a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:19.726144 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06acffdf-d798-44d4-82a9-5640e73bc8a9 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:19.740313 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-06acffdf-d798-44d4-82a9-5640e73bc8a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:41:19.740313 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:41:19.740313 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:41:19.740313 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:41:19.740313 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:41:19.740313 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:41:19.740313 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:41:19.740313 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:41:19.740313 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:41:19.740313 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:41:19.740313 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:41:19.740313 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:41:19.740313 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 12:41:19.740313 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 12:41:19.740313 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 12:41:19.741258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 12:41:19.741258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:41:19.741258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:41:19.741258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:41:19.741258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:41:19.741258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:41:19.741258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:41:19.741258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:41:19.741258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:41:19.741258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:41:19.741258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:41:19.741258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:41:19.741258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:41:19.741258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:41:19.741258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:41:19.741258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:41:19.741258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:41:19.742202 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 12:41:19.742202 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 12:41:19.742202 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 12:41:19.742202 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 12:41:19.742202 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 12:41:19.742202 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 12:41:19.742202 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:41:19.742202 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:41:19.748256 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 607/1217] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:41:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:41:19.748409 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d4f362-ef42-4532-8dd6-87ca36636651 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:19.874248 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d4f362-ef42-4532-8dd6-87ca36636651 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:19.905249 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:19.921915 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca058f-0049-4872-bc63-1cdd5fc642f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:19.923070 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca058f-0049-4872-bc63-1cdd5fc642f8 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:19.923810 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca058f-0049-4872-bc63-1cdd5fc642f8 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:19.924397 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca058f-0049-4872-bc63-1cdd5fc642f8 None admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:19.925598 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ca058f-0049-4872-bc63-1cdd5fc642f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:41:19.926722 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ca058f-0049-4872-bc63-1cdd5fc642f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:19.928476 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ca058f-0049-4872-bc63-1cdd5fc642f8 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:19.944936 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 611/1218] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 26 12:41:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:41:19.951073 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06acffdf-d798-44d4-82a9-5640e73bc8a9 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:20.079018 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06acffdf-d798-44d4-82a9-5640e73bc8a9 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:20.116546 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:20.125948 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8691d609-4374-433e-86f0-4765bda89802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:20.126883 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8691d609-4374-433e-86f0-4765bda89802 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:20.127175 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8691d609-4374-433e-86f0-4765bda89802 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:20.127452 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8691d609-4374-433e-86f0-4765bda89802 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:20.128285 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8691d609-4374-433e-86f0-4765bda89802 None admin] RBAC: Adding query filter params (role.id=9c68629183974be9bfb1ed95f69912ee, scope.project.id=644278e7e8934b37a1e7ab20d35764cd, user.id=ec49a03aed774c439459883bd5442d9d) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:41:20.128645 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8691d609-4374-433e-86f0-4765bda89802 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:20.130653 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8691d609-4374-433e-86f0-4765bda89802 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:20.147668 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 608/1219] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:41:19 2022] GET /identity/v3/role_assignments?user.id=ec49a03aed774c439459883bd5442d9d&scope.project.id=644278e7e8934b37a1e7ab20d35764cd&role.id=9c68629183974be9bfb1ed95f69912ee => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:41:22.212310 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e11f68-eafe-474c-a705-e1d55d7e0dbf None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:22.212919 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e11f68-eafe-474c-a705-e1d55d7e0dbf None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:22.212919 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e11f68-eafe-474c-a705-e1d55d7e0dbf None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:22.214295 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 612/1220] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:41:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:41:22.219689 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc81c9-1fad-4dd0-b5fd-5fc722861654 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:22.220187 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc81c9-1fad-4dd0-b5fd-5fc722861654 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:22.220291 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc81c9-1fad-4dd0-b5fd-5fc722861654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:22.239761 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-b9fc81c9-1fad-4dd0-b5fd-5fc722861654 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:41:22.241134 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-b9fc81c9-1fad-4dd0-b5fd-5fc722861654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:41:22.317782 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 609/1221] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:41:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:41:22.340285 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e11f68-eafe-474c-a705-e1d55d7e0dbf None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:22.456764 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e11f68-eafe-474c-a705-e1d55d7e0dbf None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:22.494202 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9cad0-c1dd-4ebf-a26e-42f62801afd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:22.494863 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9cad0-c1dd-4ebf-a26e-42f62801afd9 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:22.495140 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9cad0-c1dd-4ebf-a26e-42f62801afd9 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:22.495410 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9cad0-c1dd-4ebf-a26e-42f62801afd9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:22.508682 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a9cad0-c1dd-4ebf-a26e-42f62801afd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:41:22.513577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-e3a9cad0-c1dd-4ebf-a26e-42f62801afd9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 12:41:22.513577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:41:22.513577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:41:22.513577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:41:22.513577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:41:22.513577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:41:22.513577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:41:22.513577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:41:22.513577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:41:22.513577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:41:22.513577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:41:22.513577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:41:22.513577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:41:22.513577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:41:22.513577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:41:22.513577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:41:22.514524 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:41:22.514524 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:41:22.514524 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:41:22.514524 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:41:22.514524 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:41:22.514524 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:41:22.514524 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:41:22.514524 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:41:22.514524 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:41:22.514524 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:41:22.514524 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:41:22.514524 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:41:22.514524 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:41:22.514524 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:41:22.514524 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:41:22.514524 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:41:22.515447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:41:22.515447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:41:22.515447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:41:22.515447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:41:22.515447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:41:22.515447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:41:22.515447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:41:22.515447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:41:22.515447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:41:22.515447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:41:22.515447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 12:41:22.515447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:41:22.520284 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 613/1222] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 26 12:41:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:41:22.522489 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fc81c9-1fad-4dd0-b5fd-5fc722861654 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:22.628850 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fc81c9-1fad-4dd0-b5fd-5fc722861654 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:22.654444 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:22.665800 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14b11be-c384-409b-ab65-f4aa161cecef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:22.666417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b11be-c384-409b-ab65-f4aa161cecef None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:22.666700 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b11be-c384-409b-ab65-f4aa161cecef None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:22.666972 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b11be-c384-409b-ab65-f4aa161cecef None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:22.667715 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14b11be-c384-409b-ab65-f4aa161cecef None admin] RBAC: Adding query filter params (name=service) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:41:22.668021 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14b11be-c384-409b-ab65-f4aa161cecef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:22.669394 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14b11be-c384-409b-ab65-f4aa161cecef None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:22.678259 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 610/1223] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Jul 26 12:41:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:41:22.683984 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9cad0-c1dd-4ebf-a26e-42f62801afd9 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:22.731282 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:22.799863 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9cad0-c1dd-4ebf-a26e-42f62801afd9 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:22.846488 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e01d27-9666-46db-b8d4-748de7bfede2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:22.847129 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e01d27-9666-46db-b8d4-748de7bfede2 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:22.847409 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e01d27-9666-46db-b8d4-748de7bfede2 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:22.848290 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e01d27-9666-46db-b8d4-748de7bfede2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:22.859521 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e01d27-9666-46db-b8d4-748de7bfede2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:22.861203 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e01d27-9666-46db-b8d4-748de7bfede2 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:22.878194 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-82e01d27-9666-46db-b8d4-748de7bfede2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:41:22.878194 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:41:22.878194 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:41:22.878194 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:41:22.878194 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:41:22.878194 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:41:22.878194 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:41:22.878194 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:41:22.878194 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:41:22.878194 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:41:22.878194 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:41:22.878194 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:41:22.878194 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:41:22.878194 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:41:22.878194 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:41:22.884907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:41:22.884907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:41:22.884907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:41:22.884907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:41:22.884907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:41:22.884907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:41:22.884907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:41:22.884907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:41:22.884907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:41:22.884907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:41:22.884907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:41:22.884907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:41:22.884907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:41:22.884907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:41:22.884907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:41:22.884907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:41:22.884907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:41:22.886096 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:41:22.886096 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:41:22.886096 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:41:22.886096 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:41:22.886096 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 614/1224] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:41:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:41:22.886445 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14b11be-c384-409b-ab65-f4aa161cecef None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:23.000597 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14b11be-c384-409b-ab65-f4aa161cecef None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:23.038310 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3e92ce-f66b-4a46-b6fd-d9d33b57c35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:23.038923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e92ce-f66b-4a46-b6fd-d9d33b57c35b None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:23.039204 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e92ce-f66b-4a46-b6fd-d9d33b57c35b None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:23.039475 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e92ce-f66b-4a46-b6fd-d9d33b57c35b None admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:23.040195 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3e92ce-f66b-4a46-b6fd-d9d33b57c35b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:41:23.040500 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3e92ce-f66b-4a46-b6fd-d9d33b57c35b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:23.041860 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3e92ce-f66b-4a46-b6fd-d9d33b57c35b None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:23.053616 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 611/1225] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:41:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:41:23.059236 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e01d27-9666-46db-b8d4-748de7bfede2 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:23.165647 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:23.173465 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e01d27-9666-46db-b8d4-748de7bfede2 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:23.212594 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ecaa0-efc4-4d0b-90c4-0876ed76b5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:23.213336 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ecaa0-efc4-4d0b-90c4-0876ed76b5d8 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:23.213582 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ecaa0-efc4-4d0b-90c4-0876ed76b5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:23.213996 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ecaa0-efc4-4d0b-90c4-0876ed76b5d8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:23.224275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50ecaa0-efc4-4d0b-90c4-0876ed76b5d8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:23.226276 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50ecaa0-efc4-4d0b-90c4-0876ed76b5d8 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:23.240438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-a50ecaa0-efc4-4d0b-90c4-0876ed76b5d8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 12:41:23.240438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:41:23.240438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:41:23.240438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:41:23.240438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:41:23.240438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:41:23.240438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:41:23.240438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:41:23.240438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:41:23.240438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:41:23.240438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:41:23.240438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:41:23.240438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 12:41:23.240438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 12:41:23.240438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 12:41:23.240438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 12:41:23.241759 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:41:23.241759 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:41:23.241759 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 12:41:23.241759 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:41:23.241759 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 12:41:23.241759 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:41:23.241759 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:41:23.241759 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:41:23.241759 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:41:23.241759 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:41:23.241759 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:41:23.241759 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:41:23.241759 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:41:23.241759 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:41:23.241759 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:41:23.241759 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:41:23.241759 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:41:23.241759 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:41:23.243043 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:41:23.243043 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:41:23.243043 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 12:41:23.243043 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 12:41:23.243043 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 12:41:23.243043 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 12:41:23.243043 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 12:41:23.243043 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 12:41:23.243043 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 12:41:23.243043 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:41:23.247806 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 615/1226] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:41:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:41:23.250541 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3e92ce-f66b-4a46-b6fd-d9d33b57c35b None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:23.302452 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:23.363381 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3e92ce-f66b-4a46-b6fd-d9d33b57c35b None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:23.402354 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b11a29-8b10-4563-af88-5e48d435ea95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:23.403225 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b11a29-8b10-4563-af88-5e48d435ea95 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:23.403502 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b11a29-8b10-4563-af88-5e48d435ea95 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:23.403821 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b11a29-8b10-4563-af88-5e48d435ea95 None admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:23.404608 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b11a29-8b10-4563-af88-5e48d435ea95 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:41:23.404922 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b11a29-8b10-4563-af88-5e48d435ea95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:23.406346 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b11a29-8b10-4563-af88-5e48d435ea95 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:23.424838 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 612/1227] 10.222.0.20 () {56 vars in 1143 bytes} [Tue Jul 26 12:41:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:41:23.431034 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ecaa0-efc4-4d0b-90c4-0876ed76b5d8 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:23.546241 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ecaa0-efc4-4d0b-90c4-0876ed76b5d8 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:23.583101 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6caaa9-182a-4696-ba75-4291e76c3e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:23.583683 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6caaa9-182a-4696-ba75-4291e76c3e05 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:23.583961 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6caaa9-182a-4696-ba75-4291e76c3e05 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:23.584226 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6caaa9-182a-4696-ba75-4291e76c3e05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:23.596133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6caaa9-182a-4696-ba75-4291e76c3e05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:23.597828 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6caaa9-182a-4696-ba75-4291e76c3e05 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:23.614060 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-8d6caaa9-182a-4696-ba75-4291e76c3e05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:41:23.614060 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:41:23.614060 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:41:23.614060 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:41:23.614060 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:41:23.614060 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:41:23.614060 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:41:23.614060 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:41:23.614060 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:41:23.614060 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:41:23.614060 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:41:23.614060 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:41:23.614060 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:41:23.614060 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:41:23.614060 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:41:23.615181 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:41:23.615181 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:41:23.615181 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:41:23.615181 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:41:23.615181 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:41:23.615181 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:41:23.615181 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:41:23.615181 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:41:23.615181 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:41:23.615181 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:41:23.615181 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:41:23.615181 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:41:23.615181 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:41:23.615181 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:41:23.615181 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:41:23.615181 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:41:23.615181 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:41:23.616368 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:41:23.616368 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:41:23.616368 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:41:23.616368 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:41:23.620589 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 616/1228] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:41:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:41:23.621503 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b11a29-8b10-4563-af88-5e48d435ea95 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:23.732946 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b11a29-8b10-4563-af88-5e48d435ea95 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:23.735325 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:23.771848 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f900d716-da1f-4eb4-ae8f-013de68838b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:23.772472 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900d716-da1f-4eb4-ae8f-013de68838b2 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:23.772749 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900d716-da1f-4eb4-ae8f-013de68838b2 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:23.773026 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900d716-da1f-4eb4-ae8f-013de68838b2 None admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:23.773712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f900d716-da1f-4eb4-ae8f-013de68838b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:41:23.774063 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f900d716-da1f-4eb4-ae8f-013de68838b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:23.775474 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f900d716-da1f-4eb4-ae8f-013de68838b2 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:23.788485 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 613/1229] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:41:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:41:23.794191 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6caaa9-182a-4696-ba75-4291e76c3e05 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:23.804627 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:23.917728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6caaa9-182a-4696-ba75-4291e76c3e05 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:23.964178 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b136f8-ed6a-4e1e-866d-d37f26f8c3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:23.964949 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b136f8-ed6a-4e1e-866d-d37f26f8c3a1 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:23.965258 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b136f8-ed6a-4e1e-866d-d37f26f8c3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:23.965541 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b136f8-ed6a-4e1e-866d-d37f26f8c3a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:23.975314 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b136f8-ed6a-4e1e-866d-d37f26f8c3a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:23.976965 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b136f8-ed6a-4e1e-866d-d37f26f8c3a1 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:23.990489 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-d4b136f8-ed6a-4e1e-866d-d37f26f8c3a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:41:23.990489 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:41:23.990489 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:41:23.990489 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:41:23.990489 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:41:23.990489 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:41:23.990489 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:41:23.990489 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:41:23.990489 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:41:23.990489 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:41:23.990489 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:41:23.990489 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:41:23.990489 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 12:41:23.990489 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 12:41:23.990489 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 12:41:23.991494 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 12:41:23.991494 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:41:23.991494 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:41:23.991494 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:41:23.991494 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:41:23.991494 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:41:23.991494 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:41:23.991494 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:41:23.991494 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:41:23.991494 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:41:23.991494 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:41:23.991494 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:41:23.991494 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:41:23.991494 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:41:23.991494 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:41:23.991494 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:41:23.991494 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:41:23.992454 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 12:41:23.992454 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 12:41:23.992454 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 12:41:23.992454 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 12:41:23.992454 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 12:41:23.992454 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 12:41:23.992454 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:41:23.992454 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:41:23.992454 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 617/1230] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:41:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:41:23.998352 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f900d716-da1f-4eb4-ae8f-013de68838b2 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:24.106969 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f900d716-da1f-4eb4-ae8f-013de68838b2 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:24.141792 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c50726-e3cd-4e4d-aa5f-6dff293ab8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:24.142429 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c50726-e3cd-4e4d-aa5f-6dff293ab8f8 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:24.142746 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c50726-e3cd-4e4d-aa5f-6dff293ab8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:24.142962 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c50726-e3cd-4e4d-aa5f-6dff293ab8f8 None admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:24.143738 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c50726-e3cd-4e4d-aa5f-6dff293ab8f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:41:24.144029 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c50726-e3cd-4e4d-aa5f-6dff293ab8f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:24.146029 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c50726-e3cd-4e4d-aa5f-6dff293ab8f8 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:24.166760 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 614/1231] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 26 12:41:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:41:24.167986 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b136f8-ed6a-4e1e-866d-d37f26f8c3a1 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:24.241539 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:24.278418 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b136f8-ed6a-4e1e-866d-d37f26f8c3a1 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:24.315994 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effdfaa4-cb34-4dcd-a93e-0d0802d138f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:24.316743 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effdfaa4-cb34-4dcd-a93e-0d0802d138f3 None admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:24.317014 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effdfaa4-cb34-4dcd-a93e-0d0802d138f3 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:24.317292 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effdfaa4-cb34-4dcd-a93e-0d0802d138f3 None admin] PATH_INFO: `/v3/projects/644278e7e8934b37a1e7ab20d35764cd/users/ec49a03aed774c439459883bd5442d9d/roles/9c68629183974be9bfb1ed95f69912ee` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:24.333289 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effdfaa4-cb34-4dcd-a93e-0d0802d138f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=644278e7e8934b37a1e7ab20d35764cd, user_id=ec49a03aed774c439459883bd5442d9d, role_id=9c68629183974be9bfb1ed95f69912ee)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:24.336037 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effdfaa4-cb34-4dcd-a93e-0d0802d138f3 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:24.352197 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 618/1232] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 26 12:41:24 2022] PUT /identity/v3/projects/644278e7e8934b37a1e7ab20d35764cd/users/ec49a03aed774c439459883bd5442d9d/roles/9c68629183974be9bfb1ed95f69912ee => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:41:26.484146 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378c91a-7988-443a-b3de-a9ee8a4a42f2 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:26.484681 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378c91a-7988-443a-b3de-a9ee8a4a42f2 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:26.484681 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378c91a-7988-443a-b3de-a9ee8a4a42f2 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:26.486043 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 615/1233] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:41:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 12:41:26.492233 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14691c56-4e84-4636-b4c4-558629517f6f None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:26.492681 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14691c56-4e84-4636-b4c4-558629517f6f None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:26.492837 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14691c56-4e84-4636-b4c4-558629517f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:26.511055 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-14691c56-4e84-4636-b4c4-558629517f6f None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:41:26.512283 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-14691c56-4e84-4636-b4c4-558629517f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:41:26.584203 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 619/1234] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:41:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:41:26.609477 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c378c91a-7988-443a-b3de-a9ee8a4a42f2 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:26.637082 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:26.719213 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c378c91a-7988-443a-b3de-a9ee8a4a42f2 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:26.757513 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c881dcf5-31f6-4e6c-8cf8-b037c639b848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:26.758093 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881dcf5-31f6-4e6c-8cf8-b037c639b848 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:26.758369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881dcf5-31f6-4e6c-8cf8-b037c639b848 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:26.758636 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881dcf5-31f6-4e6c-8cf8-b037c639b848 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:26.769812 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c881dcf5-31f6-4e6c-8cf8-b037c639b848 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:41:26.775217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-c881dcf5-31f6-4e6c-8cf8-b037c639b848 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 12:41:26.775217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:41:26.775217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:41:26.775217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:41:26.775217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:41:26.775217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:41:26.775217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:41:26.775217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:41:26.775217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:41:26.775217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:41:26.775217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:41:26.775217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:41:26.775217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:41:26.775217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:41:26.775217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:41:26.775217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:41:26.776650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:41:26.776650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:41:26.776650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:41:26.776650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:41:26.776650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:41:26.776650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:41:26.776650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:41:26.776650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:41:26.776650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:41:26.776650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:41:26.776650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:41:26.776650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:41:26.776650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:41:26.776650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:41:26.776650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:41:26.776650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:41:26.777549 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:41:26.777549 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:41:26.777549 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:41:26.777549 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:41:26.777549 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:41:26.777549 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:41:26.777549 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:41:26.777549 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:41:26.777549 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:41:26.777549 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:41:26.777549 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 12:41:26.777549 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:41:26.777549 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 616/1235] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 26 12:41:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:41:26.783059 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14691c56-4e84-4636-b4c4-558629517f6f None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:26.836728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:26.911166 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14691c56-4e84-4636-b4c4-558629517f6f None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:26.956297 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b0ef0-4613-479c-a7d1-f99132b43da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:26.957679 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b0ef0-4613-479c-a7d1-f99132b43da8 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:26.957965 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b0ef0-4613-479c-a7d1-f99132b43da8 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:26.958238 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b0ef0-4613-479c-a7d1-f99132b43da8 None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:26.959110 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b0ef0-4613-479c-a7d1-f99132b43da8 None admin] RBAC: Adding query filter params (name=service) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:41:26.959417 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b0ef0-4613-479c-a7d1-f99132b43da8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:26.961403 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b0ef0-4613-479c-a7d1-f99132b43da8 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:26.973577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 620/1236] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Jul 26 12:41:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:41:26.979824 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c881dcf5-31f6-4e6c-8cf8-b037c639b848 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:27.071893 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:27.088221 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c881dcf5-31f6-4e6c-8cf8-b037c639b848 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:27.124283 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ff0cef-33db-42da-ad31-9fd1d135b546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:27.124914 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff0cef-33db-42da-ad31-9fd1d135b546 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:27.125332 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff0cef-33db-42da-ad31-9fd1d135b546 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:27.125663 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff0cef-33db-42da-ad31-9fd1d135b546 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:27.137782 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ff0cef-33db-42da-ad31-9fd1d135b546 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:27.139274 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ff0cef-33db-42da-ad31-9fd1d135b546 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:27.156524 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-f7ff0cef-33db-42da-ad31-9fd1d135b546 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:41:27.156524 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:41:27.156524 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:41:27.156524 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:41:27.156524 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:41:27.156524 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:41:27.156524 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:41:27.156524 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:41:27.156524 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:41:27.156524 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:41:27.156524 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:41:27.156524 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:41:27.156524 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:41:27.156524 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:41:27.156524 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:41:27.157621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:41:27.157621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:41:27.157621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:41:27.157621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:41:27.157621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:41:27.157621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:41:27.157621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:41:27.157621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:41:27.157621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:41:27.157621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:41:27.157621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:41:27.157621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:41:27.157621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:41:27.157621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:41:27.157621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:41:27.157621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:41:27.157621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:41:27.158745 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:41:27.158745 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:41:27.158745 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:41:27.158745 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:41:27.158745 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 617/1237] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:41:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:41:27.164171 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b0ef0-4613-479c-a7d1-f99132b43da8 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:27.277326 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b0ef0-4613-479c-a7d1-f99132b43da8 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:27.288124 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:27.317605 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a30a01e-fe7f-4385-9ba0-07fc279e2216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:27.318255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a30a01e-fe7f-4385-9ba0-07fc279e2216 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:27.318530 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a30a01e-fe7f-4385-9ba0-07fc279e2216 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:27.318927 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a30a01e-fe7f-4385-9ba0-07fc279e2216 None admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:27.319949 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a30a01e-fe7f-4385-9ba0-07fc279e2216 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:41:27.320402 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a30a01e-fe7f-4385-9ba0-07fc279e2216 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:27.322055 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a30a01e-fe7f-4385-9ba0-07fc279e2216 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:27.337220 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 621/1238] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:41:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:41:27.343286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ff0cef-33db-42da-ad31-9fd1d135b546 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:27.455548 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ff0cef-33db-42da-ad31-9fd1d135b546 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:27.493141 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03548962-4117-4bda-b9ef-4c032322439e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:27.494116 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03548962-4117-4bda-b9ef-4c032322439e None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:27.494433 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03548962-4117-4bda-b9ef-4c032322439e None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:27.494745 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03548962-4117-4bda-b9ef-4c032322439e None admin] PATH_INFO: `/v3/users/placement` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:27.497916 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:27.505497 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03548962-4117-4bda-b9ef-4c032322439e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:27.507010 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03548962-4117-4bda-b9ef-4c032322439e None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:27.519382 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-03548962-4117-4bda-b9ef-4c032322439e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 12:41:27.519382 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:41:27.519382 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:41:27.519382 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:41:27.519382 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:41:27.519382 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:41:27.519382 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:41:27.519382 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:41:27.519382 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:41:27.519382 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:41:27.519382 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:41:27.519382 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:41:27.519382 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 12:41:27.519382 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 12:41:27.519382 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 12:41:27.519382 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 12:41:27.520883 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:41:27.520883 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:41:27.520883 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 12:41:27.520883 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:41:27.520883 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 12:41:27.520883 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:41:27.520883 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:41:27.520883 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:41:27.520883 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:41:27.520883 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:41:27.520883 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:41:27.520883 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:41:27.520883 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:41:27.520883 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:41:27.520883 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:41:27.520883 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:41:27.520883 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:41:27.520883 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:41:27.521916 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:41:27.521916 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:41:27.521916 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 12:41:27.521916 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 12:41:27.521916 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 12:41:27.521916 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 12:41:27.521916 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 12:41:27.521916 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 12:41:27.521916 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 12:41:27.521916 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:41:27.521916 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 618/1239] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:41:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:41:27.526928 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a30a01e-fe7f-4385-9ba0-07fc279e2216 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:27.640385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a30a01e-fe7f-4385-9ba0-07fc279e2216 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:27.676688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a281c77c-26d4-4178-af20-22d8eaca06d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:27.677308 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a281c77c-26d4-4178-af20-22d8eaca06d8 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:27.677594 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a281c77c-26d4-4178-af20-22d8eaca06d8 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:27.677874 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a281c77c-26d4-4178-af20-22d8eaca06d8 None admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:27.678681 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a281c77c-26d4-4178-af20-22d8eaca06d8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:41:27.678974 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a281c77c-26d4-4178-af20-22d8eaca06d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:27.680448 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a281c77c-26d4-4178-af20-22d8eaca06d8 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:27.698258 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 622/1240] 10.222.0.20 () {56 vars in 1143 bytes} [Tue Jul 26 12:41:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:41:27.704770 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03548962-4117-4bda-b9ef-4c032322439e None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:27.814590 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03548962-4117-4bda-b9ef-4c032322439e None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:27.853883 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c0f6eb-bb91-4b34-a3c5-c45fad1d8bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:27.854667 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0f6eb-bb91-4b34-a3c5-c45fad1d8bc8 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:27.855083 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0f6eb-bb91-4b34-a3c5-c45fad1d8bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:27.855360 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0f6eb-bb91-4b34-a3c5-c45fad1d8bc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:27.927495 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:27.937000 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c0f6eb-bb91-4b34-a3c5-c45fad1d8bc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:27.939204 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c0f6eb-bb91-4b34-a3c5-c45fad1d8bc8 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:28.063026 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 619/1241] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:41:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 12:41:28.069758 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a281c77c-26d4-4178-af20-22d8eaca06d8 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:28.105585 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:28.199048 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a281c77c-26d4-4178-af20-22d8eaca06d8 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:28.238835 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0da8bf-770f-4e1b-969e-c0a61f6823d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:28.239498 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0da8bf-770f-4e1b-969e-c0a61f6823d5 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:28.239919 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0da8bf-770f-4e1b-969e-c0a61f6823d5 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:28.240200 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0da8bf-770f-4e1b-969e-c0a61f6823d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:28.254244 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0da8bf-770f-4e1b-969e-c0a61f6823d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:28.256128 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0da8bf-770f-4e1b-969e-c0a61f6823d5 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:28.274747 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-8b0da8bf-770f-4e1b-969e-c0a61f6823d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:41:28.274747 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:41:28.274747 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:41:28.274747 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:41:28.274747 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:41:28.274747 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:41:28.274747 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:41:28.274747 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:41:28.274747 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:41:28.274747 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:41:28.274747 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:41:28.274747 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:41:28.274747 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:41:28.274747 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:41:28.274747 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:41:28.275914 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:41:28.275914 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:41:28.275914 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:41:28.275914 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:41:28.275914 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:41:28.275914 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:41:28.275914 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:41:28.275914 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:41:28.275914 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:41:28.275914 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:41:28.275914 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:41:28.275914 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:41:28.275914 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:41:28.275914 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:41:28.275914 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:41:28.275914 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:41:28.275914 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:41:28.277313 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:41:28.277313 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:41:28.277313 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:41:28.277313 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:41:28.277313 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 623/1242] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:41:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:41:28.283666 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c0f6eb-bb91-4b34-a3c5-c45fad1d8bc8 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:28.398222 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:28.405497 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c0f6eb-bb91-4b34-a3c5-c45fad1d8bc8 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:28.446393 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1107d3ad-1524-409e-ae06-ba65a4aafdd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:28.446878 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1107d3ad-1524-409e-ae06-ba65a4aafdd7 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:28.447158 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1107d3ad-1524-409e-ae06-ba65a4aafdd7 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:28.447467 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1107d3ad-1524-409e-ae06-ba65a4aafdd7 None admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:28.448244 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1107d3ad-1524-409e-ae06-ba65a4aafdd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:41:28.448460 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1107d3ad-1524-409e-ae06-ba65a4aafdd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:28.449873 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1107d3ad-1524-409e-ae06-ba65a4aafdd7 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:28.462946 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 620/1243] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:41:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:41:28.469110 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0da8bf-770f-4e1b-969e-c0a61f6823d5 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:28.554273 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:28.577893 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0da8bf-770f-4e1b-969e-c0a61f6823d5 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:28.616392 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba8f2da-6cc8-439c-8999-573517416b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:28.617096 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba8f2da-6cc8-439c-8999-573517416b6e None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:28.617380 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba8f2da-6cc8-439c-8999-573517416b6e None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:28.617653 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba8f2da-6cc8-439c-8999-573517416b6e None admin] PATH_INFO: `/v3/projects/service` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:28.626086 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba8f2da-6cc8-439c-8999-573517416b6e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:28.627810 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba8f2da-6cc8-439c-8999-573517416b6e None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:28.641197 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-fba8f2da-6cc8-439c-8999-573517416b6e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:41:28.641197 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:41:28.641197 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:41:28.641197 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:41:28.641197 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:41:28.641197 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:41:28.641197 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:41:28.641197 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:41:28.641197 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:41:28.641197 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:41:28.641197 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:41:28.641197 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:41:28.641197 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 12:41:28.641197 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 12:41:28.641197 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 12:41:28.642291 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 12:41:28.642291 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:41:28.642291 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:41:28.642291 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:41:28.642291 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:41:28.642291 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:41:28.642291 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:41:28.642291 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:41:28.642291 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:41:28.642291 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:41:28.642291 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:41:28.642291 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:41:28.642291 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:41:28.642291 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:41:28.642291 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:41:28.642291 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:41:28.642291 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:41:28.643540 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 12:41:28.643540 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 12:41:28.643540 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 12:41:28.643540 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 12:41:28.643540 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 12:41:28.643540 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 12:41:28.643540 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:41:28.643540 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:41:28.643540 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 624/1244] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:41:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:41:28.648164 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1107d3ad-1524-409e-ae06-ba65a4aafdd7 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:28.766258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1107d3ad-1524-409e-ae06-ba65a4aafdd7 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:28.803809 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0f0915-07c0-41e5-b476-f2064c000124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:28.804851 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0f0915-07c0-41e5-b476-f2064c000124 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:28.805193 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0f0915-07c0-41e5-b476-f2064c000124 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:28.805523 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0f0915-07c0-41e5-b476-f2064c000124 None admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:28.806463 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0f0915-07c0-41e5-b476-f2064c000124 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:41:28.806845 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0f0915-07c0-41e5-b476-f2064c000124 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:28.808563 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0f0915-07c0-41e5-b476-f2064c000124 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:28.824119 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 621/1245] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 26 12:41:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:41:28.830307 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba8f2da-6cc8-439c-8999-573517416b6e None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:28.954653 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba8f2da-6cc8-439c-8999-573517416b6e None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:28.997983 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f86d026-0b31-47dd-90bc-8bab5dde4b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:28.998786 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f86d026-0b31-47dd-90bc-8bab5dde4b67 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:28.999111 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f86d026-0b31-47dd-90bc-8bab5dde4b67 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:28.999504 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f86d026-0b31-47dd-90bc-8bab5dde4b67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:29.000496 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f86d026-0b31-47dd-90bc-8bab5dde4b67 None admin] RBAC: Adding query filter params (role.id=9c68629183974be9bfb1ed95f69912ee, scope.project.id=644278e7e8934b37a1e7ab20d35764cd, user.id=ec49a03aed774c439459883bd5442d9d) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:41:29.000815 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f86d026-0b31-47dd-90bc-8bab5dde4b67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:29.002469 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f86d026-0b31-47dd-90bc-8bab5dde4b67 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:29.009840 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 625/1246] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:41:28 2022] GET /identity/v3/role_assignments?user.id=ec49a03aed774c439459883bd5442d9d&scope.project.id=644278e7e8934b37a1e7ab20d35764cd&role.id=9c68629183974be9bfb1ed95f69912ee => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:41:31.339024 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7c551-3350-4152-b6ef-e83e0611d38a None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:31.339576 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7c551-3350-4152-b6ef-e83e0611d38a None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:31.340240 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7c551-3350-4152-b6ef-e83e0611d38a None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:31.342589 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 622/1247] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:41:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:41:31.348639 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791c913-419a-4955-bc3e-1dce8bad15b8 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:31.349133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791c913-419a-4955-bc3e-1dce8bad15b8 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:31.349318 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791c913-419a-4955-bc3e-1dce8bad15b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:31.356799 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:31.373231 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-9791c913-419a-4955-bc3e-1dce8bad15b8 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:41:31.374671 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-9791c913-419a-4955-bc3e-1dce8bad15b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:41:31.457668 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 626/1248] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:41:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 12:41:31.485753 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7c551-3350-4152-b6ef-e83e0611d38a None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:31.507391 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:31.603934 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7c551-3350-4152-b6ef-e83e0611d38a None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:31.642856 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afd4824-045b-4eb4-b16e-faf6fe8c8b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:31.643693 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd4824-045b-4eb4-b16e-faf6fe8c8b07 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:31.643933 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd4824-045b-4eb4-b16e-faf6fe8c8b07 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:31.644241 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd4824-045b-4eb4-b16e-faf6fe8c8b07 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:31.656780 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afd4824-045b-4eb4-b16e-faf6fe8c8b07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:41:31.662458 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-5afd4824-045b-4eb4-b16e-faf6fe8c8b07 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:41:31.662458 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:41:31.662458 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:41:31.662458 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:41:31.662458 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:41:31.662458 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:41:31.662458 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:41:31.662458 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:41:31.662458 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:41:31.662458 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:41:31.662458 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:41:31.662458 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:41:31.662458 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:41:31.662458 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:41:31.662458 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:41:31.662458 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:41:31.663619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:41:31.663619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:41:31.663619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:41:31.663619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:41:31.663619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:41:31.663619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:41:31.663619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:41:31.663619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:41:31.663619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:41:31.663619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:41:31.663619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:41:31.663619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:41:31.663619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:41:31.663619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:41:31.663619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:41:31.663619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:41:31.664546 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:41:31.664546 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:41:31.664546 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:41:31.664546 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:41:31.664546 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:41:31.664546 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:41:31.664546 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:41:31.664546 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:41:31.664546 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:41:31.664546 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:41:31.664546 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:41:31.664546 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:41:31.664546 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 623/1249] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 26 12:41:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:41:31.670542 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791c913-419a-4955-bc3e-1dce8bad15b8 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:31.790750 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791c913-419a-4955-bc3e-1dce8bad15b8 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:31.827682 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:31.834616 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7542aa75-2153-4fc0-bf6a-f4ee0639d9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:31.835301 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7542aa75-2153-4fc0-bf6a-f4ee0639d9a9 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:31.835584 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7542aa75-2153-4fc0-bf6a-f4ee0639d9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:31.835883 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7542aa75-2153-4fc0-bf6a-f4ee0639d9a9 None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:31.836609 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7542aa75-2153-4fc0-bf6a-f4ee0639d9a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:41:31.836911 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7542aa75-2153-4fc0-bf6a-f4ee0639d9a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:31.838340 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7542aa75-2153-4fc0-bf6a-f4ee0639d9a9 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:31.847731 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 627/1250] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:41:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:41:31.853969 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afd4824-045b-4eb4-b16e-faf6fe8c8b07 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:31.948897 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:31.981648 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afd4824-045b-4eb4-b16e-faf6fe8c8b07 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:32.025257 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc6ec1-e4cc-4d31-bc6e-63dc0bf3f408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:32.025951 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc6ec1-e4cc-4d31-bc6e-63dc0bf3f408 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:32.026269 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc6ec1-e4cc-4d31-bc6e-63dc0bf3f408 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:32.026566 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc6ec1-e4cc-4d31-bc6e-63dc0bf3f408 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:32.039565 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edc6ec1-e4cc-4d31-bc6e-63dc0bf3f408 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:32.042047 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edc6ec1-e4cc-4d31-bc6e-63dc0bf3f408 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:32.057485 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-0edc6ec1-e4cc-4d31-bc6e-63dc0bf3f408 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:41:32.057485 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:41:32.057485 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:41:32.057485 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:41:32.057485 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:41:32.057485 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:41:32.057485 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:41:32.057485 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:41:32.057485 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:41:32.057485 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:41:32.057485 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:41:32.057485 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:41:32.057485 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:41:32.057485 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:41:32.057485 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:41:32.058657 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:41:32.058657 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:41:32.058657 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:41:32.058657 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:41:32.058657 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:41:32.058657 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:41:32.058657 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:41:32.058657 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:41:32.058657 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:41:32.058657 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:41:32.058657 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:41:32.058657 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:41:32.058657 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:41:32.058657 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:41:32.058657 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:41:32.058657 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:41:32.058657 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:41:32.059956 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:41:32.059956 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:41:32.059956 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:41:32.059956 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:41:32.059956 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 624/1251] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:41:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:41:32.064724 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7542aa75-2153-4fc0-bf6a-f4ee0639d9a9 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:32.192720 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7542aa75-2153-4fc0-bf6a-f4ee0639d9a9 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:32.232378 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e841c3a-6410-43f1-925f-660ffbb3ebee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:32.232994 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e841c3a-6410-43f1-925f-660ffbb3ebee None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:32.233303 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e841c3a-6410-43f1-925f-660ffbb3ebee None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:32.233604 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e841c3a-6410-43f1-925f-660ffbb3ebee None admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:32.234406 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e841c3a-6410-43f1-925f-660ffbb3ebee None admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:41:32.234792 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e841c3a-6410-43f1-925f-660ffbb3ebee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:32.236321 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e841c3a-6410-43f1-925f-660ffbb3ebee None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:32.250990 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 628/1252] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:41:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:41:32.257806 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc6ec1-e4cc-4d31-bc6e-63dc0bf3f408 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:32.367921 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc6ec1-e4cc-4d31-bc6e-63dc0bf3f408 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:32.396467 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:32.406106 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aad857-b07d-4133-8cd3-f399a2934e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:32.406744 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aad857-b07d-4133-8cd3-f399a2934e21 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:32.407025 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aad857-b07d-4133-8cd3-f399a2934e21 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:32.407303 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aad857-b07d-4133-8cd3-f399a2934e21 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:32.415257 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aad857-b07d-4133-8cd3-f399a2934e21 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:32.416733 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aad857-b07d-4133-8cd3-f399a2934e21 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:32.430454 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-38aad857-b07d-4133-8cd3-f399a2934e21 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 12:41:32.430454 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:41:32.430454 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:41:32.430454 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:41:32.430454 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:41:32.430454 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:41:32.430454 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:41:32.430454 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:41:32.430454 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:41:32.430454 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:41:32.430454 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:41:32.430454 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:41:32.430454 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 12:41:32.430454 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 12:41:32.430454 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 12:41:32.430454 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 12:41:32.431727 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:41:32.431727 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:41:32.431727 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 12:41:32.431727 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:41:32.431727 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 12:41:32.431727 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:41:32.431727 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:41:32.431727 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:41:32.431727 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:41:32.431727 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:41:32.431727 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:41:32.431727 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:41:32.431727 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:41:32.431727 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:41:32.431727 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:41:32.431727 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:41:32.431727 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:41:32.431727 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:41:32.432923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:41:32.432923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:41:32.432923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 12:41:32.432923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 12:41:32.432923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 12:41:32.432923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 12:41:32.432923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 12:41:32.432923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 12:41:32.432923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 12:41:32.432923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:41:32.432923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 625/1253] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:41:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:41:32.440348 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e841c3a-6410-43f1-925f-660ffbb3ebee None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:32.509629 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:32.560949 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e841c3a-6410-43f1-925f-660ffbb3ebee None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:32.600734 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0414e5c4-a29b-4f73-ab6c-a8226fc411f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:32.601347 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414e5c4-a29b-4f73-ab6c-a8226fc411f5 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:32.601625 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414e5c4-a29b-4f73-ab6c-a8226fc411f5 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:32.601896 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414e5c4-a29b-4f73-ab6c-a8226fc411f5 None admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:32.602685 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0414e5c4-a29b-4f73-ab6c-a8226fc411f5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:41:32.602991 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0414e5c4-a29b-4f73-ab6c-a8226fc411f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:32.604525 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0414e5c4-a29b-4f73-ab6c-a8226fc411f5 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:32.625532 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 629/1254] 10.222.0.20 () {56 vars in 1143 bytes} [Tue Jul 26 12:41:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:41:32.631387 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aad857-b07d-4133-8cd3-f399a2934e21 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:32.754878 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aad857-b07d-4133-8cd3-f399a2934e21 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:32.794693 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a1738f-a15e-45d1-855b-e79316eb9a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:32.795575 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a1738f-a15e-45d1-855b-e79316eb9a17 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:32.795916 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a1738f-a15e-45d1-855b-e79316eb9a17 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:32.796323 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a1738f-a15e-45d1-855b-e79316eb9a17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:32.859266 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:32.874416 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a1738f-a15e-45d1-855b-e79316eb9a17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:32.875957 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a1738f-a15e-45d1-855b-e79316eb9a17 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:33.005981 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 626/1255] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:41:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 12:41:33.012044 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0414e5c4-a29b-4f73-ab6c-a8226fc411f5 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:33.148134 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0414e5c4-a29b-4f73-ab6c-a8226fc411f5 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:33.168858 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:33.195546 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8688462e-fe4f-4eba-ae99-1f8ac51239be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:33.196247 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8688462e-fe4f-4eba-ae99-1f8ac51239be None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:33.196528 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8688462e-fe4f-4eba-ae99-1f8ac51239be None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:33.196799 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8688462e-fe4f-4eba-ae99-1f8ac51239be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:33.208170 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8688462e-fe4f-4eba-ae99-1f8ac51239be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:33.209636 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8688462e-fe4f-4eba-ae99-1f8ac51239be None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:33.226431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-8688462e-fe4f-4eba-ae99-1f8ac51239be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:41:33.226431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:41:33.226431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:41:33.226431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:41:33.226431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:41:33.226431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:41:33.226431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:41:33.226431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:41:33.226431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:41:33.226431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:41:33.226431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:41:33.226431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:41:33.226431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:41:33.226431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:41:33.226431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:41:33.227767 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:41:33.227767 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:41:33.227767 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:41:33.227767 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:41:33.227767 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:41:33.227767 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:41:33.227767 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:41:33.227767 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:41:33.227767 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:41:33.227767 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:41:33.227767 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:41:33.227767 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:41:33.227767 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:41:33.227767 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:41:33.227767 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:41:33.227767 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:41:33.227767 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:41:33.228720 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:41:33.228720 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:41:33.228720 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:41:33.228720 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:41:33.228720 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 630/1256] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:41:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 12:41:33.235527 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a1738f-a15e-45d1-855b-e79316eb9a17 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:33.332755 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:33.346153 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a1738f-a15e-45d1-855b-e79316eb9a17 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:33.383460 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32b47e4-ea16-400e-beb7-70c39ff0aeed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:33.384196 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b47e4-ea16-400e-beb7-70c39ff0aeed None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:33.384385 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b47e4-ea16-400e-beb7-70c39ff0aeed None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:33.384699 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b47e4-ea16-400e-beb7-70c39ff0aeed None admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:33.385338 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32b47e4-ea16-400e-beb7-70c39ff0aeed None admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:41:33.385640 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32b47e4-ea16-400e-beb7-70c39ff0aeed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:33.387015 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32b47e4-ea16-400e-beb7-70c39ff0aeed None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:33.398836 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 627/1257] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:41:33.404618 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8688462e-fe4f-4eba-ae99-1f8ac51239be None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:33.525720 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8688462e-fe4f-4eba-ae99-1f8ac51239be None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:33.565216 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df31395a-6b93-4f2d-a3f8-2daeef0c2eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:33.565917 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df31395a-6b93-4f2d-a3f8-2daeef0c2eb8 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:33.566233 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df31395a-6b93-4f2d-a3f8-2daeef0c2eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:33.566513 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df31395a-6b93-4f2d-a3f8-2daeef0c2eb8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:33.574210 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df31395a-6b93-4f2d-a3f8-2daeef0c2eb8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:33.575869 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df31395a-6b93-4f2d-a3f8-2daeef0c2eb8 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:33.588559 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-df31395a-6b93-4f2d-a3f8-2daeef0c2eb8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:41:33.588559 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:41:33.588559 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:41:33.588559 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:41:33.588559 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:41:33.588559 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:41:33.588559 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:41:33.588559 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:41:33.588559 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:41:33.588559 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:41:33.588559 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:41:33.588559 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:41:33.588559 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 12:41:33.588559 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 12:41:33.588559 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 12:41:33.589595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 12:41:33.589595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:41:33.589595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:41:33.589595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:41:33.589595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:41:33.589595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:41:33.589595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:41:33.589595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:41:33.589595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:41:33.589595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:41:33.589595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:41:33.589595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:41:33.589595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:41:33.589595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:41:33.589595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:41:33.589595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:41:33.589595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:41:33.590527 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 12:41:33.590527 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 12:41:33.590527 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 12:41:33.590527 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 12:41:33.590527 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 12:41:33.590527 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 12:41:33.590527 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:41:33.590527 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:41:33.590527 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 631/1258] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:41:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:41:33.596188 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32b47e4-ea16-400e-beb7-70c39ff0aeed None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:33.712650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32b47e4-ea16-400e-beb7-70c39ff0aeed None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:33.751542 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d9aca6-f855-4a28-8df5-c11854dfaed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:33.752191 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9aca6-f855-4a28-8df5-c11854dfaed9 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:33.752472 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9aca6-f855-4a28-8df5-c11854dfaed9 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:33.752740 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9aca6-f855-4a28-8df5-c11854dfaed9 None admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:33.753491 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d9aca6-f855-4a28-8df5-c11854dfaed9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:41:33.754418 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d9aca6-f855-4a28-8df5-c11854dfaed9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:33.756056 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d9aca6-f855-4a28-8df5-c11854dfaed9 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:33.778465 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 628/1259] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 26 12:41:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:41:33.780290 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df31395a-6b93-4f2d-a3f8-2daeef0c2eb8 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:33.819354 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:33.920079 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df31395a-6b93-4f2d-a3f8-2daeef0c2eb8 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:33.958672 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1899783-dc3e-4171-967c-e0b84ea42f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:33.959238 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1899783-dc3e-4171-967c-e0b84ea42f39 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:33.959503 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1899783-dc3e-4171-967c-e0b84ea42f39 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:33.959822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1899783-dc3e-4171-967c-e0b84ea42f39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:33.960628 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1899783-dc3e-4171-967c-e0b84ea42f39 None admin] RBAC: Adding query filter params (role.id=0e78444e43c744668178971af4087ca0, scope.project.id=644278e7e8934b37a1e7ab20d35764cd, user.id=ec49a03aed774c439459883bd5442d9d) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:41:33.960908 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1899783-dc3e-4171-967c-e0b84ea42f39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:33.962339 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1899783-dc3e-4171-967c-e0b84ea42f39 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:33.968687 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 632/1260] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:41:33 2022] GET /identity/v3/role_assignments?user.id=ec49a03aed774c439459883bd5442d9d&scope.project.id=644278e7e8934b37a1e7ab20d35764cd&role.id=0e78444e43c744668178971af4087ca0 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:41:35.999757 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86985007-5514-416b-8431-1c0c0f842620 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:36.000359 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86985007-5514-416b-8431-1c0c0f842620 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:36.000359 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86985007-5514-416b-8431-1c0c0f842620 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:36.001680 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 629/1261] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:41:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:41:36.007260 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c9b5bb-2712-479e-925b-2ede54f027de None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:36.007692 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c9b5bb-2712-479e-925b-2ede54f027de None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:36.007876 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c9b5bb-2712-479e-925b-2ede54f027de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:36.027479 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-52c9b5bb-2712-479e-925b-2ede54f027de None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:41:36.028776 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-52c9b5bb-2712-479e-925b-2ede54f027de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:41:36.098581 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 633/1262] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:41:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:41:36.121512 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86985007-5514-416b-8431-1c0c0f842620 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:36.132976 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:36.224155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86985007-5514-416b-8431-1c0c0f842620 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:36.257767 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7212951b-3f43-4f8e-bab7-f73e251be307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:36.258327 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212951b-3f43-4f8e-bab7-f73e251be307 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:36.258605 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212951b-3f43-4f8e-bab7-f73e251be307 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:36.258876 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212951b-3f43-4f8e-bab7-f73e251be307 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:36.268929 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7212951b-3f43-4f8e-bab7-f73e251be307 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:41:36.273888 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-7212951b-3f43-4f8e-bab7-f73e251be307 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:41:36.273888 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:41:36.273888 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:41:36.273888 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:41:36.273888 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:41:36.273888 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:41:36.273888 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:41:36.273888 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:41:36.273888 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:41:36.273888 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:41:36.273888 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:41:36.273888 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:41:36.273888 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:41:36.273888 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:41:36.273888 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:41:36.273888 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:41:36.275006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:41:36.275006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:41:36.275006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:41:36.275006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:41:36.275006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:41:36.275006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:41:36.275006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:41:36.275006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:41:36.275006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:41:36.275006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:41:36.275006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:41:36.275006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:41:36.275006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:41:36.275006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:41:36.275006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:41:36.275006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:41:36.276102 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:41:36.276102 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:41:36.276102 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:41:36.276102 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:41:36.276102 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:41:36.276102 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:41:36.276102 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:41:36.276102 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:41:36.276102 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:41:36.276102 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:41:36.276102 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:41:36.276102 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:41:36.276102 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 630/1263] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 26 12:41:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:41:36.281139 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c9b5bb-2712-479e-925b-2ede54f027de None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:36.298795 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:36.376853 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c9b5bb-2712-479e-925b-2ede54f027de None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:36.409649 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78771557-21bd-4622-ad94-6df001bb3810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:36.410276 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78771557-21bd-4622-ad94-6df001bb3810 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:36.410552 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78771557-21bd-4622-ad94-6df001bb3810 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:36.410822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78771557-21bd-4622-ad94-6df001bb3810 None admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:36.411528 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78771557-21bd-4622-ad94-6df001bb3810 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:41:36.411858 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78771557-21bd-4622-ad94-6df001bb3810 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:36.413223 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78771557-21bd-4622-ad94-6df001bb3810 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:36.421904 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 634/1264] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:41:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:41:36.430115 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7212951b-3f43-4f8e-bab7-f73e251be307 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:36.498997 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:36.533324 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7212951b-3f43-4f8e-bab7-f73e251be307 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:36.568962 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c9356-a5de-45fa-9182-84572ceaed6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:36.569647 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c9356-a5de-45fa-9182-84572ceaed6d None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:36.569933 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c9356-a5de-45fa-9182-84572ceaed6d None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:36.570211 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c9356-a5de-45fa-9182-84572ceaed6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:36.580574 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2c9356-a5de-45fa-9182-84572ceaed6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:36.582098 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2c9356-a5de-45fa-9182-84572ceaed6d None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:36.597932 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-9d2c9356-a5de-45fa-9182-84572ceaed6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:41:36.597932 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:41:36.597932 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:41:36.597932 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:41:36.597932 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:41:36.597932 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:41:36.597932 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:41:36.597932 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:41:36.597932 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:41:36.597932 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:41:36.597932 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:41:36.597932 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:41:36.597932 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:41:36.597932 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:41:36.597932 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:41:36.598949 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:41:36.598949 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:41:36.598949 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:41:36.598949 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:41:36.598949 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:41:36.598949 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:41:36.598949 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:41:36.598949 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:41:36.598949 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:41:36.598949 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:41:36.598949 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:41:36.598949 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:41:36.598949 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:41:36.598949 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:41:36.598949 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:41:36.598949 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:41:36.598949 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:41:36.600135 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:41:36.600135 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:41:36.600135 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:41:36.600135 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:41:36.600135 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 631/1265] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:41:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:41:36.605251 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78771557-21bd-4622-ad94-6df001bb3810 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:36.686252 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:36.705637 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78771557-21bd-4622-ad94-6df001bb3810 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:36.739044 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e898289-a935-4b28-af84-78f26341e8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:36.739655 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e898289-a935-4b28-af84-78f26341e8e7 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:36.739956 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e898289-a935-4b28-af84-78f26341e8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:36.740230 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e898289-a935-4b28-af84-78f26341e8e7 None admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:36.740912 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e898289-a935-4b28-af84-78f26341e8e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:41:36.741218 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e898289-a935-4b28-af84-78f26341e8e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:36.742587 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e898289-a935-4b28-af84-78f26341e8e7 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:36.754446 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 635/1266] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:41:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:41:36.760273 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c9356-a5de-45fa-9182-84572ceaed6d None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:36.875276 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c9356-a5de-45fa-9182-84572ceaed6d None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:36.896215 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:36.911630 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdbd3d9-a705-42cb-97f2-b902db34db67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:36.912358 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbd3d9-a705-42cb-97f2-b902db34db67 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:36.912679 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbd3d9-a705-42cb-97f2-b902db34db67 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:36.912978 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbd3d9-a705-42cb-97f2-b902db34db67 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:36.920897 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdbd3d9-a705-42cb-97f2-b902db34db67 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:36.922479 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdbd3d9-a705-42cb-97f2-b902db34db67 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:36.934730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-5cdbd3d9-a705-42cb-97f2-b902db34db67 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 12:41:36.934730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:41:36.934730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:41:36.934730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:41:36.934730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:41:36.934730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:41:36.934730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:41:36.934730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:41:36.934730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:41:36.934730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:41:36.934730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:41:36.934730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:41:36.934730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 12:41:36.934730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 12:41:36.934730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 12:41:36.934730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 12:41:36.935936 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:41:36.935936 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:41:36.935936 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 12:41:36.935936 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:41:36.935936 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 12:41:36.935936 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:41:36.935936 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:41:36.935936 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:41:36.935936 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:41:36.935936 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:41:36.935936 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:41:36.935936 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:41:36.935936 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:41:36.935936 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:41:36.935936 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:41:36.935936 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:41:36.935936 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:41:36.935936 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:41:36.937362 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:41:36.937362 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:41:36.937362 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 12:41:36.937362 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 12:41:36.937362 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 12:41:36.937362 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 12:41:36.937362 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 12:41:36.937362 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 12:41:36.937362 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 12:41:36.937362 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:41:36.937362 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 632/1267] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:41:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:41:36.945798 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e898289-a935-4b28-af84-78f26341e8e7 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:37.058518 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e898289-a935-4b28-af84-78f26341e8e7 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:37.091471 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e3e1f-4aa4-4673-875d-3ecef6672ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:37.092097 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e3e1f-4aa4-4673-875d-3ecef6672ea0 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:37.092379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e3e1f-4aa4-4673-875d-3ecef6672ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:37.092648 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e3e1f-4aa4-4673-875d-3ecef6672ea0 None admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:37.093431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60e3e1f-4aa4-4673-875d-3ecef6672ea0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:41:37.093737 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60e3e1f-4aa4-4673-875d-3ecef6672ea0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:37.095166 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60e3e1f-4aa4-4673-875d-3ecef6672ea0 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:37.104288 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:37.113537 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 636/1268] 10.222.0.20 () {56 vars in 1143 bytes} [Tue Jul 26 12:41:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:41:37.119453 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdbd3d9-a705-42cb-97f2-b902db34db67 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:37.217807 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdbd3d9-a705-42cb-97f2-b902db34db67 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:37.250044 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dca09c3-f0f7-4f78-8271-54d854062e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:37.250661 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca09c3-f0f7-4f78-8271-54d854062e9a None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:37.250941 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca09c3-f0f7-4f78-8271-54d854062e9a None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:37.251210 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca09c3-f0f7-4f78-8271-54d854062e9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:37.261786 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dca09c3-f0f7-4f78-8271-54d854062e9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:37.263235 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dca09c3-f0f7-4f78-8271-54d854062e9a None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:37.277586 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-1dca09c3-f0f7-4f78-8271-54d854062e9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:41:37.277586 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:41:37.277586 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:41:37.277586 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:41:37.277586 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:41:37.277586 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:41:37.277586 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:41:37.277586 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:41:37.277586 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:41:37.277586 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:41:37.277586 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:41:37.277586 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:41:37.277586 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:41:37.277586 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:41:37.277586 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:41:37.278626 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:41:37.278626 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:41:37.278626 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:41:37.278626 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:41:37.278626 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:41:37.278626 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:41:37.278626 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:41:37.278626 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:41:37.278626 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:41:37.278626 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:41:37.278626 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:41:37.278626 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:41:37.278626 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:41:37.278626 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:41:37.278626 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:41:37.278626 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:41:37.278626 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:41:37.279577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:41:37.279577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:41:37.279577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:41:37.279577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:41:37.279577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 633/1269] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:41:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:41:37.284407 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e3e1f-4aa4-4673-875d-3ecef6672ea0 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:37.382078 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e3e1f-4aa4-4673-875d-3ecef6672ea0 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:37.415256 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8693ee9c-59f9-47fe-be3e-6879f08b866b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:37.415885 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693ee9c-59f9-47fe-be3e-6879f08b866b None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:37.416166 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693ee9c-59f9-47fe-be3e-6879f08b866b None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:37.416437 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693ee9c-59f9-47fe-be3e-6879f08b866b None admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:37.417118 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8693ee9c-59f9-47fe-be3e-6879f08b866b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:41:37.417420 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8693ee9c-59f9-47fe-be3e-6879f08b866b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:37.418786 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8693ee9c-59f9-47fe-be3e-6879f08b866b None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:37.429764 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 637/1270] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:41:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:41:37.435313 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dca09c3-f0f7-4f78-8271-54d854062e9a None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:37.466680 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:37.538423 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dca09c3-f0f7-4f78-8271-54d854062e9a None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:37.571827 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81107f4c-16f3-47c2-994a-cf259c50a4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:37.572451 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81107f4c-16f3-47c2-994a-cf259c50a4bf None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:37.572727 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81107f4c-16f3-47c2-994a-cf259c50a4bf None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:37.572997 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81107f4c-16f3-47c2-994a-cf259c50a4bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:37.579579 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81107f4c-16f3-47c2-994a-cf259c50a4bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:37.581033 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81107f4c-16f3-47c2-994a-cf259c50a4bf None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:37.592759 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-81107f4c-16f3-47c2-994a-cf259c50a4bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:41:37.592759 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:41:37.592759 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:41:37.592759 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:41:37.592759 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:41:37.592759 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:41:37.592759 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:41:37.592759 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:41:37.592759 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:41:37.592759 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:41:37.592759 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:41:37.592759 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:41:37.592759 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 12:41:37.592759 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 12:41:37.592759 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 12:41:37.593767 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 12:41:37.593767 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:41:37.593767 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:41:37.593767 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:41:37.593767 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:41:37.593767 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:41:37.593767 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:41:37.593767 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:41:37.593767 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:41:37.593767 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:41:37.593767 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:41:37.593767 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:41:37.593767 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:41:37.593767 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:41:37.593767 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:41:37.593767 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:41:37.593767 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:41:37.595072 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 12:41:37.595072 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 12:41:37.595072 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 12:41:37.595072 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 12:41:37.595072 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 12:41:37.595072 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 12:41:37.595072 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:41:37.595072 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:41:37.595072 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 634/1271] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:41:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:41:37.599771 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8693ee9c-59f9-47fe-be3e-6879f08b866b None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:37.660513 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:37.705422 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8693ee9c-59f9-47fe-be3e-6879f08b866b None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:37.739316 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a498c8-8b0e-4f7e-8129-7c885ea8df19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:37.739964 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a498c8-8b0e-4f7e-8129-7c885ea8df19 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:37.740285 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a498c8-8b0e-4f7e-8129-7c885ea8df19 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:37.740587 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a498c8-8b0e-4f7e-8129-7c885ea8df19 None admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:37.741391 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a498c8-8b0e-4f7e-8129-7c885ea8df19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:41:37.741700 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a498c8-8b0e-4f7e-8129-7c885ea8df19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:37.743266 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a498c8-8b0e-4f7e-8129-7c885ea8df19 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:37.756553 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 638/1272] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 26 12:41:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:41:37.762631 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81107f4c-16f3-47c2-994a-cf259c50a4bf None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:37.856244 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:37.865099 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81107f4c-16f3-47c2-994a-cf259c50a4bf None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:37.899782 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b3369-f398-4ec3-a8f8-3949705550d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:37.900599 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b3369-f398-4ec3-a8f8-3949705550d6 None admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:37.900921 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b3369-f398-4ec3-a8f8-3949705550d6 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:37.901197 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b3369-f398-4ec3-a8f8-3949705550d6 None admin] PATH_INFO: `/v3/projects/644278e7e8934b37a1e7ab20d35764cd/users/ec49a03aed774c439459883bd5442d9d/roles/0e78444e43c744668178971af4087ca0` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:37.903141 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7b3369-f398-4ec3-a8f8-3949705550d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=644278e7e8934b37a1e7ab20d35764cd, user_id=ec49a03aed774c439459883bd5442d9d, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:37.904609 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7b3369-f398-4ec3-a8f8-3949705550d6 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:37.919082 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 635/1273] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 26 12:41:37 2022] PUT /identity/v3/projects/644278e7e8934b37a1e7ab20d35764cd/users/ec49a03aed774c439459883bd5442d9d/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:41:39.893133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464341e7-d23f-4a10-ad25-7c00dc6e7444 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:39.893715 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464341e7-d23f-4a10-ad25-7c00dc6e7444 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:39.893715 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464341e7-d23f-4a10-ad25-7c00dc6e7444 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:39.895091 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 639/1274] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:41:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:41:39.901229 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1779ab1f-97c5-412a-b379-cdf19de3ecd8 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:39.901703 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1779ab1f-97c5-412a-b379-cdf19de3ecd8 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:39.901813 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1779ab1f-97c5-412a-b379-cdf19de3ecd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:39.919692 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-1779ab1f-97c5-412a-b379-cdf19de3ecd8 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:41:39.920892 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-1779ab1f-97c5-412a-b379-cdf19de3ecd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:41:39.998261 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 636/1275] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:41:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:41:40.021869 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464341e7-d23f-4a10-ad25-7c00dc6e7444 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:40.121694 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464341e7-d23f-4a10-ad25-7c00dc6e7444 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:40.138966 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:40.155217 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8766b9c3-f2c8-4432-925f-b280b5d9fc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:40.155799 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766b9c3-f2c8-4432-925f-b280b5d9fc8e None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:40.156075 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766b9c3-f2c8-4432-925f-b280b5d9fc8e None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:40.156342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766b9c3-f2c8-4432-925f-b280b5d9fc8e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:40.166399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8766b9c3-f2c8-4432-925f-b280b5d9fc8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 12:41:40.171237 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-8766b9c3-f2c8-4432-925f-b280b5d9fc8e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:41:40.171237 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:41:40.171237 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:41:40.171237 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:41:40.171237 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:41:40.171237 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:41:40.171237 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:41:40.171237 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:41:40.171237 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:41:40.171237 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:41:40.171237 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:41:40.171237 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:41:40.171237 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 12:41:40.171237 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 12:41:40.171237 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 12:41:40.171237 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 12:41:40.172353 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 12:41:40.172353 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 12:41:40.172353 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 12:41:40.172353 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 12:41:40.172353 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:41:40.172353 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:41:40.172353 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:41:40.172353 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:41:40.172353 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:41:40.172353 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:41:40.172353 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:41:40.172353 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:41:40.172353 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:41:40.172353 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:41:40.172353 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:41:40.172353 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:41:40.173415 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:41:40.173415 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:41:40.173415 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:41:40.173415 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:41:40.173415 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 12:41:40.173415 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 12:41:40.173415 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 12:41:40.173415 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 12:41:40.173415 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 12:41:40.173415 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 12:41:40.173415 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 12:41:40.173415 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:41:40.173415 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 640/1276] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 26 12:41:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:41:40.179424 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1779ab1f-97c5-412a-b379-cdf19de3ecd8 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:40.258067 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:40.276109 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1779ab1f-97c5-412a-b379-cdf19de3ecd8 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:40.309227 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd19ec-c298-4e7d-a3b8-6136d951e943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:40.309835 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd19ec-c298-4e7d-a3b8-6136d951e943 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:40.310114 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd19ec-c298-4e7d-a3b8-6136d951e943 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:40.310383 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd19ec-c298-4e7d-a3b8-6136d951e943 None admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:40.311090 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd19ec-c298-4e7d-a3b8-6136d951e943 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:41:40.311391 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd19ec-c298-4e7d-a3b8-6136d951e943 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:40.312778 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd19ec-c298-4e7d-a3b8-6136d951e943 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:40.320624 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 637/1277] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 26 12:41:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:41:40.326431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8766b9c3-f2c8-4432-925f-b280b5d9fc8e None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:40.423251 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8766b9c3-f2c8-4432-925f-b280b5d9fc8e None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:40.455905 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b665a47-6634-43c2-ae73-51064cb7f228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:40.456525 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b665a47-6634-43c2-ae73-51064cb7f228 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:40.456835 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b665a47-6634-43c2-ae73-51064cb7f228 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:40.457108 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b665a47-6634-43c2-ae73-51064cb7f228 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:40.466784 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b665a47-6634-43c2-ae73-51064cb7f228 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:40.468262 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b665a47-6634-43c2-ae73-51064cb7f228 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:40.482916 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-8b665a47-6634-43c2-ae73-51064cb7f228 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:41:40.482916 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:41:40.482916 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:41:40.482916 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:41:40.482916 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:41:40.482916 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:41:40.482916 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:41:40.482916 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:41:40.482916 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:41:40.482916 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:41:40.482916 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:41:40.482916 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:41:40.482916 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:41:40.482916 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:41:40.482916 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:41:40.484004 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:41:40.484004 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:41:40.484004 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:41:40.484004 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:41:40.484004 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:41:40.484004 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:41:40.484004 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:41:40.484004 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:41:40.484004 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:41:40.484004 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:41:40.484004 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:41:40.484004 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:41:40.484004 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:41:40.484004 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:41:40.484004 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:41:40.484004 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:41:40.484004 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:41:40.485283 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:41:40.485283 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:41:40.485283 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:41:40.485283 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:41:40.485283 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 641/1278] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:41:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:41:40.490464 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd19ec-c298-4e7d-a3b8-6136d951e943 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:40.591253 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd19ec-c298-4e7d-a3b8-6136d951e943 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:40.624215 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e4754-56df-47ea-acb4-024dd14195e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:40.624823 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e4754-56df-47ea-acb4-024dd14195e4 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:40.625106 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e4754-56df-47ea-acb4-024dd14195e4 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:40.625403 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e4754-56df-47ea-acb4-024dd14195e4 None admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:40.626092 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4e4754-56df-47ea-acb4-024dd14195e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:41:40.626393 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4e4754-56df-47ea-acb4-024dd14195e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:40.627787 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4e4754-56df-47ea-acb4-024dd14195e4 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:40.639228 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 638/1279] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:41:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:41:40.645648 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b665a47-6634-43c2-ae73-51064cb7f228 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:40.676480 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:40.744993 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b665a47-6634-43c2-ae73-51064cb7f228 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:40.777413 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c4cc2b-5efc-49cc-bd9d-b483c3fe9719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:40.778042 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c4cc2b-5efc-49cc-bd9d-b483c3fe9719 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:40.778324 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c4cc2b-5efc-49cc-bd9d-b483c3fe9719 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:40.778598 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c4cc2b-5efc-49cc-bd9d-b483c3fe9719 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:40.786741 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c4cc2b-5efc-49cc-bd9d-b483c3fe9719 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:40.788219 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c4cc2b-5efc-49cc-bd9d-b483c3fe9719 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:40.799188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-79c4cc2b-5efc-49cc-bd9d-b483c3fe9719 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 12:41:40.799188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:41:40.799188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:41:40.799188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:41:40.799188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:41:40.799188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:41:40.799188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:41:40.799188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:41:40.799188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:41:40.799188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:41:40.799188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:41:40.799188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:41:40.799188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 12:41:40.799188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 12:41:40.799188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 12:41:40.799188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 12:41:40.800394 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:41:40.800394 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:41:40.800394 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 12:41:40.800394 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:41:40.800394 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 12:41:40.800394 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 12:41:40.800394 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:41:40.800394 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:41:40.800394 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:41:40.800394 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:41:40.800394 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:41:40.800394 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:41:40.800394 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:41:40.800394 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:41:40.800394 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:41:40.800394 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:41:40.800394 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:41:40.800394 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:41:40.801896 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:41:40.801896 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:41:40.801896 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 12:41:40.801896 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 12:41:40.801896 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 12:41:40.801896 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 12:41:40.801896 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 12:41:40.801896 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 12:41:40.801896 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 12:41:40.801896 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:41:40.801896 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 642/1280] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:41:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:41:40.806347 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e4754-56df-47ea-acb4-024dd14195e4 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:40.810105 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:40.903065 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e4754-56df-47ea-acb4-024dd14195e4 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:40.936642 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077dbcd-e274-4383-a97a-9b5255be282f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:40.937184 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077dbcd-e274-4383-a97a-9b5255be282f None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:40.937467 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077dbcd-e274-4383-a97a-9b5255be282f None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:40.937738 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077dbcd-e274-4383-a97a-9b5255be282f None admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:40.938514 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6077dbcd-e274-4383-a97a-9b5255be282f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:41:40.938817 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6077dbcd-e274-4383-a97a-9b5255be282f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:40.940298 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6077dbcd-e274-4383-a97a-9b5255be282f None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:40.957447 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 639/1281] 10.222.0.20 () {56 vars in 1143 bytes} [Tue Jul 26 12:41:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:41:40.963101 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c4cc2b-5efc-49cc-bd9d-b483c3fe9719 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:41.059091 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:41.069252 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c4cc2b-5efc-49cc-bd9d-b483c3fe9719 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:41.101883 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddbecfd-9262-4aa7-bec8-0e136e8fb2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:41.102484 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddbecfd-9262-4aa7-bec8-0e136e8fb2b0 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:41.102763 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddbecfd-9262-4aa7-bec8-0e136e8fb2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:41.103032 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddbecfd-9262-4aa7-bec8-0e136e8fb2b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:41.168664 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddbecfd-9262-4aa7-bec8-0e136e8fb2b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:41.170126 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddbecfd-9262-4aa7-bec8-0e136e8fb2b0 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:41.257433 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:41.267804 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 643/1282] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 26 12:41:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 12:41:41.274156 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077dbcd-e274-4383-a97a-9b5255be282f None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:41.333557 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:41.371116 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077dbcd-e274-4383-a97a-9b5255be282f None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:41.403410 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7add2f28-5aed-4dfb-8ff4-80fddc414291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:41.404060 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add2f28-5aed-4dfb-8ff4-80fddc414291 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:41.404339 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add2f28-5aed-4dfb-8ff4-80fddc414291 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:41.404608 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add2f28-5aed-4dfb-8ff4-80fddc414291 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:41.414246 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7add2f28-5aed-4dfb-8ff4-80fddc414291 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:41.415699 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7add2f28-5aed-4dfb-8ff4-80fddc414291 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:41.430203 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-7add2f28-5aed-4dfb-8ff4-80fddc414291 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:41:41.430203 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:41:41.430203 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:41:41.430203 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:41:41.430203 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:41:41.430203 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:41:41.430203 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:41:41.430203 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:41:41.430203 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:41:41.430203 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:41:41.430203 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:41:41.430203 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:41:41.430203 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 12:41:41.430203 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 12:41:41.430203 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 12:41:41.431297 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 12:41:41.431297 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:41:41.431297 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:41:41.431297 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:41:41.431297 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:41:41.431297 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:41:41.431297 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:41:41.431297 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:41:41.431297 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:41:41.431297 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:41:41.431297 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:41:41.431297 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:41:41.431297 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:41:41.431297 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:41:41.431297 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:41:41.431297 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:41:41.431297 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:41:41.432299 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 12:41:41.432299 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 12:41:41.432299 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 12:41:41.432299 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:41:41.432299 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 640/1283] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:41:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:41:41.437022 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddbecfd-9262-4aa7-bec8-0e136e8fb2b0 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:41.535407 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddbecfd-9262-4aa7-bec8-0e136e8fb2b0 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:41.567701 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03caced-fe09-403c-8f9e-acb4c59bf1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:41.568304 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03caced-fe09-403c-8f9e-acb4c59bf1c3 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:41.568580 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03caced-fe09-403c-8f9e-acb4c59bf1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:41.568853 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03caced-fe09-403c-8f9e-acb4c59bf1c3 None admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:41.569534 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03caced-fe09-403c-8f9e-acb4c59bf1c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:41:41.569836 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03caced-fe09-403c-8f9e-acb4c59bf1c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:41.571201 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03caced-fe09-403c-8f9e-acb4c59bf1c3 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:41.582708 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 644/1284] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 26 12:41:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:41:41.588385 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7add2f28-5aed-4dfb-8ff4-80fddc414291 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:41.687544 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7add2f28-5aed-4dfb-8ff4-80fddc414291 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:41.700736 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:41.723275 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b89885-8ca5-4239-b60d-06bbd6b5ba92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:41.723949 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b89885-8ca5-4239-b60d-06bbd6b5ba92 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:41.724228 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b89885-8ca5-4239-b60d-06bbd6b5ba92 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:41.724499 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b89885-8ca5-4239-b60d-06bbd6b5ba92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:41.731301 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b89885-8ca5-4239-b60d-06bbd6b5ba92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:41.732781 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b89885-8ca5-4239-b60d-06bbd6b5ba92 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:41.744270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application [None req-f2b89885-8ca5-4239-b60d-06bbd6b5ba92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:41:41.744270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:41:41.744270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:41:41.744270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:41:41.744270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:41:41.744270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:41:41.744270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:41:41.744270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:41:41.744270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:41:41.744270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:41:41.744270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:41:41.744270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:41:41.744270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 12:41:41.744270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 12:41:41.744270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 12:41:41.745332 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 12:41:41.745332 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:41:41.745332 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:41:41.745332 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:41:41.745332 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:41:41.745332 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:41:41.745332 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:41:41.745332 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:41:41.745332 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application with Lock( Jul 26 12:41:41.745332 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:41:41.745332 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:41:41.745332 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:41:41.745332 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:41:41.745332 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:41:41.745332 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:41:41.745332 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:41:41.745332 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:41:41.746289 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 12:41:41.746289 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 12:41:41.746289 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 12:41:41.746289 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 12:41:41.746289 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 12:41:41.746289 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 12:41:41.746289 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 12:41:41.746289 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: ERROR keystone.server.flask.application Jul 26 12:41:41.746289 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 641/1285] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 26 12:41:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:41:41.751262 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03caced-fe09-403c-8f9e-acb4c59bf1c3 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:41.801118 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:41.850673 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03caced-fe09-403c-8f9e-acb4c59bf1c3 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:41.882944 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d6cbf9-a141-4316-bb4a-604e9626504e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:41.883572 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6cbf9-a141-4316-bb4a-604e9626504e None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:41.883944 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6cbf9-a141-4316-bb4a-604e9626504e None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:41.884212 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6cbf9-a141-4316-bb4a-604e9626504e None admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:41.884940 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d6cbf9-a141-4316-bb4a-604e9626504e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:41:41.885242 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d6cbf9-a141-4316-bb4a-604e9626504e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:41.886654 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d6cbf9-a141-4316-bb4a-604e9626504e None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:41.899951 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 645/1286] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 26 12:41:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:41:41.906093 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b89885-8ca5-4239-b60d-06bbd6b5ba92 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:42.004259 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b89885-8ca5-4239-b60d-06bbd6b5ba92 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:42.037031 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef95ada-3910-4b56-b780-f2332cfc5058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:42.037637 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef95ada-3910-4b56-b780-f2332cfc5058 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:42.037921 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef95ada-3910-4b56-b780-f2332cfc5058 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:42.038199 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef95ada-3910-4b56-b780-f2332cfc5058 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:42.038978 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef95ada-3910-4b56-b780-f2332cfc5058 None admin] RBAC: Adding query filter params (role.id=0e78444e43c744668178971af4087ca0, scope.project.id=644278e7e8934b37a1e7ab20d35764cd, user.id=ec49a03aed774c439459883bd5442d9d) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:41:42.039288 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef95ada-3910-4b56-b780-f2332cfc5058 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:42.040749 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef95ada-3910-4b56-b780-f2332cfc5058 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:42.046370 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 642/1287] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 26 12:41:41 2022] GET /identity/v3/role_assignments?user.id=ec49a03aed774c439459883bd5442d9d&scope.project.id=644278e7e8934b37a1e7ab20d35764cd&role.id=0e78444e43c744668178971af4087ca0 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:41:44.023915 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec03899-5978-4c4d-89f4-162ab63514c5 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:44.024446 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec03899-5978-4c4d-89f4-162ab63514c5 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:44.024446 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec03899-5978-4c4d-89f4-162ab63514c5 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:44.025815 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 646/1288] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:41:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:41:44.030873 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc679ff-ab40-46dc-be1f-4d5444d153c1 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:44.031276 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc679ff-ab40-46dc-be1f-4d5444d153c1 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:44.031418 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc679ff-ab40-46dc-be1f-4d5444d153c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:44.049453 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-dcc679ff-ab40-46dc-be1f-4d5444d153c1 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:41:44.051239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-dcc679ff-ab40-46dc-be1f-4d5444d153c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:41:44.079507 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:44.116775 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 643/1289] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:41:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:41:44.138866 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec03899-5978-4c4d-89f4-162ab63514c5 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:44.241338 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec03899-5978-4c4d-89f4-162ab63514c5 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:44.251888 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:44.274443 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba8f36e-cd76-4157-874f-f11ef4764a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:44.275004 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8f36e-cd76-4157-874f-f11ef4764a0f None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:44.275285 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8f36e-cd76-4157-874f-f11ef4764a0f None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:44.275558 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8f36e-cd76-4157-874f-f11ef4764a0f None admin] PATH_INFO: `/v3/services/placement` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:44.276206 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba8f36e-cd76-4157-874f-f11ef4764a0f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:44.277594 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba8f36e-cd76-4157-874f-f11ef4764a0f None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:44.287255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-cba8f36e-cd76-4157-874f-f11ef4764a0f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 12:41:44.287255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:41:44.287255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 12:41:44.287255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 12:41:44.287255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 12:41:44.287255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 12:41:44.287255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 12:41:44.287255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:41:44.287255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 12:41:44.287255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:41:44.287255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 12:41:44.287255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:41:44.287255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:41:44.287255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:41:44.287255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:41:44.287255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:41:44.287255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:41:44.288534 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:41:44.288534 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:41:44.288534 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:41:44.288534 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:41:44.288534 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:41:44.288534 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:41:44.288534 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 12:41:44.288534 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 12:41:44.288534 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 12:41:44.288534 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 12:41:44.288534 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:41:44.288534 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:41:44.288534 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:41:44.288534 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:41:44.288534 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:41:44.288534 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:41:44.289585 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:41:44.289585 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:41:44.289585 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:41:44.289585 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:41:44.289585 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:41:44.289585 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:41:44.289585 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:41:44.289585 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:41:44.289585 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:41:44.289585 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:41:44.289585 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 12:41:44.289585 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:41:44.289585 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 12:41:44.289585 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:41:44.289585 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 647/1290] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Jul 26 12:41:44 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:41:44.294454 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc679ff-ab40-46dc-be1f-4d5444d153c1 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:44.390893 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc679ff-ab40-46dc-be1f-4d5444d153c1 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:44.424373 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e85c0-9b55-4c94-890e-2ac5bb98745a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:44.424980 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e85c0-9b55-4c94-890e-2ac5bb98745a None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:44.425266 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e85c0-9b55-4c94-890e-2ac5bb98745a None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:44.425561 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e85c0-9b55-4c94-890e-2ac5bb98745a None admin] PATH_INFO: `/v3/services` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:44.426226 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106e85c0-9b55-4c94-890e-2ac5bb98745a None admin] RBAC: Adding query filter params (name=placement) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:41:44.426530 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106e85c0-9b55-4c94-890e-2ac5bb98745a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:44.427924 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106e85c0-9b55-4c94-890e-2ac5bb98745a None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:44.432650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 644/1291] 10.222.0.20 () {56 vars in 1116 bytes} [Tue Jul 26 12:41:44 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:41:44.438384 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba8f36e-cd76-4157-874f-f11ef4764a0f None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:44.538281 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba8f36e-cd76-4157-874f-f11ef4764a0f None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:44.570257 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274579bd-6d98-4c4c-a402-b8538ed3d44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:44.570853 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274579bd-6d98-4c4c-a402-b8538ed3d44c None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:44.571134 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274579bd-6d98-4c4c-a402-b8538ed3d44c None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:44.571410 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274579bd-6d98-4c4c-a402-b8538ed3d44c None admin] PATH_INFO: `/v3/services` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:44.572113 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274579bd-6d98-4c4c-a402-b8538ed3d44c None admin] RBAC: Adding query filter params (type=placement) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:41:44.572418 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274579bd-6d98-4c4c-a402-b8538ed3d44c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:44.573780 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274579bd-6d98-4c4c-a402-b8538ed3d44c None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:44.578093 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 648/1292] 10.222.0.20 () {56 vars in 1116 bytes} [Tue Jul 26 12:41:44 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:41:46.589314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b97625-5c44-4028-826e-f948bee1faa3 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:46.589846 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b97625-5c44-4028-826e-f948bee1faa3 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:46.589846 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b97625-5c44-4028-826e-f948bee1faa3 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:46.591229 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 645/1293] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:41:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:41:46.596511 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b4a5d-6970-47b4-a65a-484afc6226b1 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:46.596924 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b4a5d-6970-47b4-a65a-484afc6226b1 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:46.597061 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b4a5d-6970-47b4-a65a-484afc6226b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:46.615466 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-a98b4a5d-6970-47b4-a65a-484afc6226b1 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:41:46.616674 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-a98b4a5d-6970-47b4-a65a-484afc6226b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:41:46.645768 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:46.683020 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 649/1294] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:41:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:41:46.705138 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b97625-5c44-4028-826e-f948bee1faa3 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:46.736449 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:46.806766 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b97625-5c44-4028-826e-f948bee1faa3 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:46.844595 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0609f3-ebdc-40d9-818c-82d72c991a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:46.845120 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0609f3-ebdc-40d9-818c-82d72c991a18 None admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:46.845402 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0609f3-ebdc-40d9-818c-82d72c991a18 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:46.845677 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0609f3-ebdc-40d9-818c-82d72c991a18 None admin] PATH_INFO: `/v3/services` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:46.846388 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0609f3-ebdc-40d9-818c-82d72c991a18 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:46.847826 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0609f3-ebdc-40d9-818c-82d72c991a18 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:46.862331 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 646/1295] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Jul 26 12:41:46 2022] POST /identity/v3/services => generated 246 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:41:48.900028 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c302b-38ff-4f24-b8a0-e09284414579 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:48.900571 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c302b-38ff-4f24-b8a0-e09284414579 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:48.900571 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c302b-38ff-4f24-b8a0-e09284414579 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:48.901914 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 650/1296] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:41:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:41:48.907075 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d781e9-bf7a-410a-baaf-9ae4f94299ab None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:48.907506 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d781e9-bf7a-410a-baaf-9ae4f94299ab None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:48.907632 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d781e9-bf7a-410a-baaf-9ae4f94299ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:48.925933 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-68d781e9-bf7a-410a-baaf-9ae4f94299ab None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:41:48.927121 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-68d781e9-bf7a-410a-baaf-9ae4f94299ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:41:49.000562 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 647/1297] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:41:48 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:41:49.023320 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c302b-38ff-4f24-b8a0-e09284414579 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:49.126972 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c302b-38ff-4f24-b8a0-e09284414579 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:49.167883 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e504ecdb-17dc-46da-92dc-dd11daae2faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:49.168611 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504ecdb-17dc-46da-92dc-dd11daae2faf None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:49.168897 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504ecdb-17dc-46da-92dc-dd11daae2faf None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:49.169174 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504ecdb-17dc-46da-92dc-dd11daae2faf None admin] PATH_INFO: `/v3/services/placement` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:49.169815 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e504ecdb-17dc-46da-92dc-dd11daae2faf None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:49.171195 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e504ecdb-17dc-46da-92dc-dd11daae2faf None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:49.181573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-e504ecdb-17dc-46da-92dc-dd11daae2faf None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 12:41:49.181573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:41:49.181573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 12:41:49.181573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 12:41:49.181573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 12:41:49.181573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 12:41:49.181573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 12:41:49.181573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:41:49.181573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 12:41:49.181573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:41:49.181573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 12:41:49.181573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:41:49.181573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:41:49.181573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:41:49.181573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:41:49.181573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:41:49.181573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:41:49.182740 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:41:49.182740 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:41:49.182740 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:41:49.182740 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:41:49.182740 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:41:49.182740 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:41:49.182740 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 12:41:49.182740 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 12:41:49.182740 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 12:41:49.182740 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 12:41:49.182740 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:41:49.182740 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:41:49.182740 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:41:49.182740 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:41:49.182740 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:41:49.182740 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:41:49.183790 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:41:49.183790 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:41:49.183790 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:41:49.183790 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:41:49.183790 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:41:49.183790 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:41:49.183790 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:41:49.183790 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:41:49.183790 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:41:49.183790 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:41:49.183790 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 12:41:49.183790 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:41:49.183790 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 12:41:49.183790 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:41:49.183790 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 651/1298] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Jul 26 12:41:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:41:49.190533 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d781e9-bf7a-410a-baaf-9ae4f94299ab None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:49.226166 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:49.307230 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d781e9-bf7a-410a-baaf-9ae4f94299ab None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:49.340115 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e0208-1479-4a0c-91d8-1e48e7d3b036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:49.340727 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e0208-1479-4a0c-91d8-1e48e7d3b036 None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:49.341008 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e0208-1479-4a0c-91d8-1e48e7d3b036 None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:49.341283 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e0208-1479-4a0c-91d8-1e48e7d3b036 None admin] PATH_INFO: `/v3/services` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:49.341953 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959e0208-1479-4a0c-91d8-1e48e7d3b036 None admin] RBAC: Adding query filter params (name=placement) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:41:49.342259 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959e0208-1479-4a0c-91d8-1e48e7d3b036 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:49.343665 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959e0208-1479-4a0c-91d8-1e48e7d3b036 None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:49.349189 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 648/1299] 10.222.0.20 () {56 vars in 1116 bytes} [Tue Jul 26 12:41:49 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:41:49.355214 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e504ecdb-17dc-46da-92dc-dd11daae2faf None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:49.386884 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:49.453240 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e504ecdb-17dc-46da-92dc-dd11daae2faf None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:49.485298 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399a5cda-e12d-46b7-800b-06335028079b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:49.485900 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a5cda-e12d-46b7-800b-06335028079b None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:49.486177 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a5cda-e12d-46b7-800b-06335028079b None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:49.486449 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a5cda-e12d-46b7-800b-06335028079b None admin] PATH_INFO: `/v3/endpoints` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:49.487191 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399a5cda-e12d-46b7-800b-06335028079b None admin] RBAC: Adding query filter params (interface=public, service_id=593cc6c36ff1484a9b6f2b423170059d, region_id=RegionOne) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:41:49.487496 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399a5cda-e12d-46b7-800b-06335028079b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:49.489901 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399a5cda-e12d-46b7-800b-06335028079b None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:49.495077 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 652/1300] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Jul 26 12:41:49 2022] GET /identity/v3/endpoints?service_id=593cc6c36ff1484a9b6f2b423170059d&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:41:49.501125 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e0208-1479-4a0c-91d8-1e48e7d3b036 None admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:49.584638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:49.602935 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e0208-1479-4a0c-91d8-1e48e7d3b036 None admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:49.635091 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e6c72-14ff-4b51-8722-d0ec93acaf8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:49.635712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e6c72-14ff-4b51-8722-d0ec93acaf8e None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:49.635991 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e6c72-14ff-4b51-8722-d0ec93acaf8e None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:49.636257 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e6c72-14ff-4b51-8722-d0ec93acaf8e None admin] PATH_INFO: `/v3/services` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:49.636897 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e6c72-14ff-4b51-8722-d0ec93acaf8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:49.638250 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e6c72-14ff-4b51-8722-d0ec93acaf8e None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:49.645480 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 649/1301] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Jul 26 12:41:49 2022] GET /identity/v3/services => generated 2191 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:41:51.656291 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a86da07-337f-41e2-9ca1-875e27aa4fe8 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:51.656813 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a86da07-337f-41e2-9ca1-875e27aa4fe8 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:51.656930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a86da07-337f-41e2-9ca1-875e27aa4fe8 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:51.658196 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 653/1302] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:41:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:41:51.663923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cadbea2-7c6b-4100-ba04-0bdffb701e8e None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:51.664349 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cadbea2-7c6b-4100-ba04-0bdffb701e8e None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:51.664473 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cadbea2-7c6b-4100-ba04-0bdffb701e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:51.682653 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-3cadbea2-7c6b-4100-ba04-0bdffb701e8e None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:41:51.683858 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-3cadbea2-7c6b-4100-ba04-0bdffb701e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:41:51.750165 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 650/1303] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:41:51 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 12:41:51.772610 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a86da07-337f-41e2-9ca1-875e27aa4fe8 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:51.875893 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:51.888552 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a86da07-337f-41e2-9ca1-875e27aa4fe8 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:51.923825 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db983d14-0ce2-4e52-90c1-d852d8761225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:51.924376 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db983d14-0ce2-4e52-90c1-d852d8761225 None admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:51.924660 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db983d14-0ce2-4e52-90c1-d852d8761225 None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:51.924937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db983d14-0ce2-4e52-90c1-d852d8761225 None admin] PATH_INFO: `/v3/services/placement` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:51.925594 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db983d14-0ce2-4e52-90c1-d852d8761225 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:51.926963 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db983d14-0ce2-4e52-90c1-d852d8761225 None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:51.935728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application [None req-db983d14-0ce2-4e52-90c1-d852d8761225 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 12:41:51.935728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:41:51.935728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 12:41:51.935728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 12:41:51.935728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 12:41:51.935728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 12:41:51.935728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 12:41:51.935728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:41:51.935728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 12:41:51.935728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:41:51.935728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 12:41:51.935728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:41:51.935728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 12:41:51.935728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 12:41:51.935728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 12:41:51.935728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 12:41:51.935728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 12:41:51.936904 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 12:41:51.936904 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 12:41:51.936904 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 12:41:51.936904 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 12:41:51.936904 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 12:41:51.936904 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 12:41:51.936904 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 12:41:51.936904 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 12:41:51.936904 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 12:41:51.936904 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 12:41:51.936904 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 12:41:51.936904 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 12:41:51.936904 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 12:41:51.936904 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 12:41:51.936904 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 12:41:51.936904 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 12:41:51.937933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 12:41:51.937933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application with Lock( Jul 26 12:41:51.937933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 12:41:51.937933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self._enter() Jul 26 12:41:51.937933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 12:41:51.937933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 12:41:51.937933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 12:41:51.937933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application return self.creator() Jul 26 12:41:51.937933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 12:41:51.937933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application created_value = creator( Jul 26 12:41:51.937933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 12:41:51.937933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 12:41:51.937933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 12:41:51.937933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: ERROR keystone.server.flask.application Jul 26 12:41:51.937933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 654/1304] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Jul 26 12:41:51 2022] GET /identity/v3/services/placement => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 12:41:51.944190 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cadbea2-7c6b-4100-ba04-0bdffb701e8e None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:52.027802 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:41:52.050546 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cadbea2-7c6b-4100-ba04-0bdffb701e8e None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:52.082802 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99744ee-3f53-42ae-92ec-5af2e300d62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:52.083407 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99744ee-3f53-42ae-92ec-5af2e300d62f None admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:52.083707 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99744ee-3f53-42ae-92ec-5af2e300d62f None admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:52.083982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99744ee-3f53-42ae-92ec-5af2e300d62f None admin] PATH_INFO: `/v3/services` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:52.084644 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99744ee-3f53-42ae-92ec-5af2e300d62f None admin] RBAC: Adding query filter params (name=placement) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:41:52.084942 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99744ee-3f53-42ae-92ec-5af2e300d62f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:52.086296 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99744ee-3f53-42ae-92ec-5af2e300d62f None admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:52.091700 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 651/1305] 10.222.0.20 () {56 vars in 1116 bytes} [Tue Jul 26 12:41:51 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:41:52.098145 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db983d14-0ce2-4e52-90c1-d852d8761225 None admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:41:52.195615 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db983d14-0ce2-4e52-90c1-d852d8761225 None admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:41:52.228709 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d864810-346a-4b95-b3c2-c14d2eaea02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:41:52.229301 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d864810-346a-4b95-b3c2-c14d2eaea02d None admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:41:52.229576 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d864810-346a-4b95-b3c2-c14d2eaea02d None admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:41:52.229842 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d864810-346a-4b95-b3c2-c14d2eaea02d None admin] PATH_INFO: `/v3/endpoints` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:41:52.230549 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d864810-346a-4b95-b3c2-c14d2eaea02d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:41:52.231927 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d864810-346a-4b95-b3c2-c14d2eaea02d None admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:41:52.251189 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 655/1306] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:41:52 2022] POST /identity/v3/endpoints => generated 331 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:42:02.088942 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=30 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:42:02.246756 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:42:55.421724 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b4d12-d9ba-4feb-85bd-747c00c7f3df None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:42:55.422309 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b4d12-d9ba-4feb-85bd-747c00c7f3df None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:42:55.422309 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b4d12-d9ba-4feb-85bd-747c00c7f3df None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:42:55.429041 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394cc593-189c-46df-80ab-fb70e5ce7756 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:42:55.429568 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394cc593-189c-46df-80ab-fb70e5ce7756 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:42:55.429568 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394cc593-189c-46df-80ab-fb70e5ce7756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:42:55.434412 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 652/1307] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:42:55 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:42:55.507342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-394cc593-189c-46df-80ab-fb70e5ce7756 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:42:55.508519 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-394cc593-189c-46df-80ab-fb70e5ce7756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:42:55.617023 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 656/1308] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:42:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:42:55.674354 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4bcf5-5e73-43bf-83d2-e25fc8f36273 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:42:55.677184 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4bcf5-5e73-43bf-83d2-e25fc8f36273 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:42:55.678404 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4bcf5-5e73-43bf-83d2-e25fc8f36273 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:42:55.685398 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 653/1309] 10.222.0.20 () {54 vars in 965 bytes} [Tue Jul 26 12:42:55 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:42:55.692624 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be0084-ffbc-4d91-bd52-652f2d472e6f None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:42:55.692916 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be0084-ffbc-4d91-bd52-652f2d472e6f None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:42:55.693193 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be0084-ffbc-4d91-bd52-652f2d472e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:42:55.805713 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-a6be0084-ffbc-4d91-bd52-652f2d472e6f None None] MFA Rules not processed for user `6830350c6eba4610bbe9ab1dc31f5939`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:42:55.809062 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-a6be0084-ffbc-4d91-bd52-652f2d472e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:42:55.861658 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:42:55.934227 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 657/1310] 10.222.0.20 () {58 vars in 1062 bytes} [Tue Jul 26 12:42:55 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:42:55.944264 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def4bcf5-5e73-43bf-83d2-e25fc8f36273 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:42:55.946312 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-def4bcf5-5e73-43bf-83d2-e25fc8f36273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:42:56.071213 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def4bcf5-5e73-43bf-83d2-e25fc8f36273 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:42:56.152346 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355313eb-f320-441b-a96f-f959c82bbf2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:42:56.153004 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355313eb-f320-441b-a96f-f959c82bbf2a service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:42:56.153378 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355313eb-f320-441b-a96f-f959c82bbf2a service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:42:56.153566 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355313eb-f320-441b-a96f-f959c82bbf2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:42:56.155182 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-355313eb-f320-441b-a96f-f959c82bbf2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:42:56.220316 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:42:56.280669 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355313eb-f320-441b-a96f-f959c82bbf2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:42:56.282119 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355313eb-f320-441b-a96f-f959c82bbf2a service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:42:56.301496 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 654/1311] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:42:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:42:58.610508 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e6917-04cd-4cbb-a08b-9d5394bf7dc0 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:42:58.611047 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e6917-04cd-4cbb-a08b-9d5394bf7dc0 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:42:58.611047 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e6917-04cd-4cbb-a08b-9d5394bf7dc0 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:42:58.612945 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 658/1312] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:42:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:42:58.618154 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd309e5e-a9b1-4f58-ab3c-08b06acac83f None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:42:58.618571 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd309e5e-a9b1-4f58-ab3c-08b06acac83f None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:42:58.618734 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd309e5e-a9b1-4f58-ab3c-08b06acac83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:42:58.700627 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-fd309e5e-a9b1-4f58-ab3c-08b06acac83f None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:42:58.701853 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-fd309e5e-a9b1-4f58-ab3c-08b06acac83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:42:58.800867 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 655/1313] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:42:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:42:58.843486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e6917-04cd-4cbb-a08b-9d5394bf7dc0 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:42:58.845059 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-444e6917-04cd-4cbb-a08b-9d5394bf7dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:42:58.963755 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e6917-04cd-4cbb-a08b-9d5394bf7dc0 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:42:59.042990 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:42:59.045592 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c2e69-a82f-49d8-acb3-d7c01b2404ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:42:59.046217 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c2e69-a82f-49d8-acb3-d7c01b2404ef service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:42:59.046760 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c2e69-a82f-49d8-acb3-d7c01b2404ef service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:42:59.047260 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c2e69-a82f-49d8-acb3-d7c01b2404ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:42:59.049147 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-e83c2e69-a82f-49d8-acb3-d7c01b2404ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:42:59.164525 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83c2e69-a82f-49d8-acb3-d7c01b2404ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:42:59.165980 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83c2e69-a82f-49d8-acb3-d7c01b2404ef service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:42:59.186271 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 659/1314] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:42:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:43:01.514128 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280adcd-16ae-41af-95f4-f0d833e1ca6e None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:01.514640 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280adcd-16ae-41af-95f4-f0d833e1ca6e None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:01.514640 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280adcd-16ae-41af-95f4-f0d833e1ca6e None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:01.516026 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 656/1315] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:43:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:43:01.521184 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a156c0-7685-4907-9fcd-8ac53e076709 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:01.521592 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a156c0-7685-4907-9fcd-8ac53e076709 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:01.521736 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a156c0-7685-4907-9fcd-8ac53e076709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:01.589338 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-a7a156c0-7685-4907-9fcd-8ac53e076709 None None] MFA Rules not processed for user `8fecfbe4ea8e4bd29589fe594487625c`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:43:01.590704 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-a7a156c0-7685-4907-9fcd-8ac53e076709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:01.646324 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:43:01.692440 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 660/1316] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:43:01 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:43:01.733257 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9280adcd-16ae-41af-95f4-f0d833e1ca6e None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:43:01.734817 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-9280adcd-16ae-41af-95f4-f0d833e1ca6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:01.744739 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:43:01.841696 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9280adcd-16ae-41af-95f4-f0d833e1ca6e None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:43:01.918286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c614bbc-c17f-46d2-b7a4-46ce813e8a8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:43:01.918903 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c614bbc-c17f-46d2-b7a4-46ce813e8a8e service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:01.919178 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c614bbc-c17f-46d2-b7a4-46ce813e8a8e service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:01.919445 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c614bbc-c17f-46d2-b7a4-46ce813e8a8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:01.921095 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-4c614bbc-c17f-46d2-b7a4-46ce813e8a8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:01.985306 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:43:02.032318 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c614bbc-c17f-46d2-b7a4-46ce813e8a8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:43:02.033760 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c614bbc-c17f-46d2-b7a4-46ce813e8a8e service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:43:02.051844 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 657/1317] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:43:01 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:43:04.637940 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c29d5-8c57-45b6-b250-1e01fa815c74 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:04.638480 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c29d5-8c57-45b6-b250-1e01fa815c74 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:04.638480 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c29d5-8c57-45b6-b250-1e01fa815c74 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:04.639859 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 661/1318] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:43:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:43:04.645045 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4905fe3-616b-4f67-bbde-fa83dfd8dac3 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:04.645514 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4905fe3-616b-4f67-bbde-fa83dfd8dac3 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:04.645607 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4905fe3-616b-4f67-bbde-fa83dfd8dac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:04.711597 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-b4905fe3-616b-4f67-bbde-fa83dfd8dac3 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:43:04.712819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-b4905fe3-616b-4f67-bbde-fa83dfd8dac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:04.809073 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 658/1319] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:43:04 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:43:04.848137 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c29d5-8c57-45b6-b250-1e01fa815c74 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:43:04.849682 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-496c29d5-8c57-45b6-b250-1e01fa815c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:04.956130 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c29d5-8c57-45b6-b250-1e01fa815c74 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:43:05.028243 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb52533d-bbf5-4b76-9c7e-7cf83ccc471b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:43:05.028849 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52533d-bbf5-4b76-9c7e-7cf83ccc471b service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:05.029128 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52533d-bbf5-4b76-9c7e-7cf83ccc471b service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:05.029399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52533d-bbf5-4b76-9c7e-7cf83ccc471b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:05.030991 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-eb52533d-bbf5-4b76-9c7e-7cf83ccc471b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:05.038778 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:43:05.140209 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb52533d-bbf5-4b76-9c7e-7cf83ccc471b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:43:05.141657 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb52533d-bbf5-4b76-9c7e-7cf83ccc471b service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:43:05.159873 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 662/1320] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:43:04 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:43:07.841089 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa28c3b7-29ac-4e8d-9b86-966d9aba336b None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:07.841684 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa28c3b7-29ac-4e8d-9b86-966d9aba336b None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:07.841684 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa28c3b7-29ac-4e8d-9b86-966d9aba336b None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:07.843006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 659/1321] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:43:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:43:07.848323 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1befa5c-bc28-4918-81b5-120b21795a42 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:07.848746 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1befa5c-bc28-4918-81b5-120b21795a42 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:07.848909 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1befa5c-bc28-4918-81b5-120b21795a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:07.923175 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-f1befa5c-bc28-4918-81b5-120b21795a42 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:43:07.924426 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-f1befa5c-bc28-4918-81b5-120b21795a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:07.986278 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:43:08.030588 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 663/1322] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:43:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:43:08.081467 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7806bd6-8fbc-48f1-b51b-a6871348dff1 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:08.081753 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7806bd6-8fbc-48f1-b51b-a6871348dff1 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:08.082029 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7806bd6-8fbc-48f1-b51b-a6871348dff1 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:08.083421 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 660/1323] 10.222.0.20 () {54 vars in 965 bytes} [Tue Jul 26 12:43:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:43:08.089896 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d71373-3f4d-49a8-9209-73ff20a31816 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:08.090195 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d71373-3f4d-49a8-9209-73ff20a31816 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:08.090529 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d71373-3f4d-49a8-9209-73ff20a31816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:08.174412 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-d8d71373-3f4d-49a8-9209-73ff20a31816 None None] MFA Rules not processed for user `6830350c6eba4610bbe9ab1dc31f5939`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:43:08.176231 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-d8d71373-3f4d-49a8-9209-73ff20a31816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:08.259598 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 664/1324] 10.222.0.20 () {58 vars in 1062 bytes} [Tue Jul 26 12:43:08 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:43:08.268843 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7806bd6-8fbc-48f1-b51b-a6871348dff1 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:43:08.270407 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-a7806bd6-8fbc-48f1-b51b-a6871348dff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:08.290735 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:43:08.376385 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7806bd6-8fbc-48f1-b51b-a6871348dff1 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:43:08.449898 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a52308-a696-4e8d-b4c5-651ee1b0480f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:43:08.450410 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a52308-a696-4e8d-b4c5-651ee1b0480f service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:08.450692 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a52308-a696-4e8d-b4c5-651ee1b0480f service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:08.450966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a52308-a696-4e8d-b4c5-651ee1b0480f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:08.452633 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-a5a52308-a696-4e8d-b4c5-651ee1b0480f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:08.525154 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:43:08.563992 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a52308-a696-4e8d-b4c5-651ee1b0480f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:43:08.565494 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a52308-a696-4e8d-b4c5-651ee1b0480f service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:43:08.584488 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 661/1325] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:43:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 12:43:10.870533 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949a9d2-29af-4a83-a14f-884970e30f4b None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:10.871149 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949a9d2-29af-4a83-a14f-884970e30f4b None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:10.871149 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949a9d2-29af-4a83-a14f-884970e30f4b None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:10.872705 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 665/1326] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:43:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 12:43:10.878322 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7667c4b-a51a-46d9-916a-70a8f3390fc2 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:10.878794 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7667c4b-a51a-46d9-916a-70a8f3390fc2 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:10.878871 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7667c4b-a51a-46d9-916a-70a8f3390fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:10.954687 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-d7667c4b-a51a-46d9-916a-70a8f3390fc2 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:43:10.956440 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-d7667c4b-a51a-46d9-916a-70a8f3390fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:11.057186 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 662/1327] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:43:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:43:11.093640 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9949a9d2-29af-4a83-a14f-884970e30f4b None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:43:11.095099 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-9949a9d2-29af-4a83-a14f-884970e30f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:11.169437 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:43:11.216132 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9949a9d2-29af-4a83-a14f-884970e30f4b None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:43:11.291958 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d701e-c0f9-473c-9610-ad621a28b703 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:43:11.292577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d701e-c0f9-473c-9610-ad621a28b703 service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:11.292857 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d701e-c0f9-473c-9610-ad621a28b703 service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:11.293127 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d701e-c0f9-473c-9610-ad621a28b703 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:11.294749 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-d14d701e-c0f9-473c-9610-ad621a28b703 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:11.408776 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:43:11.411118 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14d701e-c0f9-473c-9610-ad621a28b703 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:43:11.412901 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14d701e-c0f9-473c-9610-ad621a28b703 service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:43:11.432802 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 666/1328] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:43:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:43:13.501903 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d998983-044c-4167-a1da-14b3eb3c7477 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:13.502487 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d998983-044c-4167-a1da-14b3eb3c7477 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:13.502487 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d998983-044c-4167-a1da-14b3eb3c7477 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:13.504371 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 663/1329] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:43:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:43:13.509596 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15560523-4618-4df2-89ce-2ba0e388e03e None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:13.510038 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15560523-4618-4df2-89ce-2ba0e388e03e None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:13.510138 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15560523-4618-4df2-89ce-2ba0e388e03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:13.578063 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-15560523-4618-4df2-89ce-2ba0e388e03e None None] MFA Rules not processed for user `8fecfbe4ea8e4bd29589fe594487625c`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:43:13.579205 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-15560523-4618-4df2-89ce-2ba0e388e03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:13.678254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 667/1330] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:43:13 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:43:13.715124 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d998983-044c-4167-a1da-14b3eb3c7477 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:43:13.716689 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-4d998983-044c-4167-a1da-14b3eb3c7477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:13.760006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:43:13.845995 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d998983-044c-4167-a1da-14b3eb3c7477 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:43:13.923601 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0fea53-5f44-4cd8-a273-e99bc8c8945e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:43:13.924242 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0fea53-5f44-4cd8-a273-e99bc8c8945e service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:13.924544 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0fea53-5f44-4cd8-a273-e99bc8c8945e service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:13.924821 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0fea53-5f44-4cd8-a273-e99bc8c8945e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:13.926436 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-8d0fea53-5f44-4cd8-a273-e99bc8c8945e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:14.006374 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:43:14.038916 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0fea53-5f44-4cd8-a273-e99bc8c8945e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:43:14.040377 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0fea53-5f44-4cd8-a273-e99bc8c8945e service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:43:14.058983 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 664/1331] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:43:13 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:43:16.330290 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe83d0-6cf8-4d39-b305-09c1458507c0 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:16.330900 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe83d0-6cf8-4d39-b305-09c1458507c0 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:16.330900 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe83d0-6cf8-4d39-b305-09c1458507c0 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:16.332290 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 668/1332] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:43:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:43:16.338620 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac751a3-6b60-4aa7-8daa-f94dab5cfafe None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:16.339130 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac751a3-6b60-4aa7-8daa-f94dab5cfafe None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:16.339628 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac751a3-6b60-4aa7-8daa-f94dab5cfafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:16.404946 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-4ac751a3-6b60-4aa7-8daa-f94dab5cfafe None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:43:16.406150 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-4ac751a3-6b60-4aa7-8daa-f94dab5cfafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:16.503252 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 665/1333] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:43:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:43:16.539551 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe83d0-6cf8-4d39-b305-09c1458507c0 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:43:16.541116 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-71fe83d0-6cf8-4d39-b305-09c1458507c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:16.635667 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:43:16.671831 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe83d0-6cf8-4d39-b305-09c1458507c0 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:43:16.744827 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4faa98-ce24-4d30-b99d-65683333a168 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:43:16.745436 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4faa98-ce24-4d30-b99d-65683333a168 service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:16.745717 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4faa98-ce24-4d30-b99d-65683333a168 service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:16.745992 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4faa98-ce24-4d30-b99d-65683333a168 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:16.747941 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-6f4faa98-ce24-4d30-b99d-65683333a168 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:16.861075 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4faa98-ce24-4d30-b99d-65683333a168 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:43:16.862438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4faa98-ce24-4d30-b99d-65683333a168 service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:43:16.872777 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:43:16.881180 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 669/1334] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:43:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:43:18.935104 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b9ee1-8aa4-4cec-8ccc-6e43530f9f98 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:18.935663 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b9ee1-8aa4-4cec-8ccc-6e43530f9f98 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:18.935663 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b9ee1-8aa4-4cec-8ccc-6e43530f9f98 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:18.937035 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 666/1335] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:43:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:43:18.942279 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1e111-9fc7-43d6-bc99-acebcde76a0c None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:18.942676 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1e111-9fc7-43d6-bc99-acebcde76a0c None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:18.942837 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1e111-9fc7-43d6-bc99-acebcde76a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:19.014155 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-97c1e111-9fc7-43d6-bc99-acebcde76a0c None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:43:19.015383 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-97c1e111-9fc7-43d6-bc99-acebcde76a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:19.113124 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 670/1336] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:43:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:43:19.154122 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4b9ee1-8aa4-4cec-8ccc-6e43530f9f98 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:43:19.155717 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-0f4b9ee1-8aa4-4cec-8ccc-6e43530f9f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:19.196564 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:43:19.260890 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4b9ee1-8aa4-4cec-8ccc-6e43530f9f98 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:43:19.336785 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e07c0-a969-4271-88cd-852e7f2232b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:43:19.337405 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e07c0-a969-4271-88cd-852e7f2232b3 service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:19.337723 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e07c0-a969-4271-88cd-852e7f2232b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:19.337950 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e07c0-a969-4271-88cd-852e7f2232b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:19.339586 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-8b2e07c0-a969-4271-88cd-852e7f2232b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:19.422530 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:43:19.448555 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2e07c0-a969-4271-88cd-852e7f2232b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:43:19.449918 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2e07c0-a969-4271-88cd-852e7f2232b3 service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:43:19.469814 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 667/1337] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:43:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:43:21.865362 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5502fe7-d83a-420c-a011-43806130ea7d None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:21.866008 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5502fe7-d83a-420c-a011-43806130ea7d None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:21.866008 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5502fe7-d83a-420c-a011-43806130ea7d None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:21.867301 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 671/1338] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:43:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:43:21.872763 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1375c60-aa76-4239-8782-f73619655ed8 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:21.873191 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1375c60-aa76-4239-8782-f73619655ed8 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:21.873306 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1375c60-aa76-4239-8782-f73619655ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:21.938708 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-b1375c60-aa76-4239-8782-f73619655ed8 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:43:21.939958 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-b1375c60-aa76-4239-8782-f73619655ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:22.042692 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 668/1339] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:43:21 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:43:22.079693 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5502fe7-d83a-420c-a011-43806130ea7d None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:43:22.081251 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-d5502fe7-d83a-420c-a011-43806130ea7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:22.157595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:43:22.187040 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5502fe7-d83a-420c-a011-43806130ea7d None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:43:22.261891 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b12a7fc-32dc-4bd2-aeb2-2fec6cec535f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:43:22.261891 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12a7fc-32dc-4bd2-aeb2-2fec6cec535f service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:22.261891 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12a7fc-32dc-4bd2-aeb2-2fec6cec535f service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:22.261891 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12a7fc-32dc-4bd2-aeb2-2fec6cec535f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:22.263271 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-5b12a7fc-32dc-4bd2-aeb2-2fec6cec535f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:22.390624 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b12a7fc-32dc-4bd2-aeb2-2fec6cec535f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:43:22.393471 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b12a7fc-32dc-4bd2-aeb2-2fec6cec535f service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:43:22.427052 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 672/1340] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:43:22 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:43:26.730582 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ecb27a-f5fe-46d6-926b-1df9f64243b0 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:26.731087 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ecb27a-f5fe-46d6-926b-1df9f64243b0 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:26.731169 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ecb27a-f5fe-46d6-926b-1df9f64243b0 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:26.733020 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 669/1341] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:43:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:43:26.738824 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873384e6-db8f-4ba5-b9b8-ac688d8809eb None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:26.739280 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873384e6-db8f-4ba5-b9b8-ac688d8809eb None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:26.739475 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873384e6-db8f-4ba5-b9b8-ac688d8809eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:26.744965 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:43:26.809251 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-873384e6-db8f-4ba5-b9b8-ac688d8809eb None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:43:26.810478 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-873384e6-db8f-4ba5-b9b8-ac688d8809eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:26.911236 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 673/1342] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:43:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:43:26.951050 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ecb27a-f5fe-46d6-926b-1df9f64243b0 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:43:26.952764 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-92ecb27a-f5fe-46d6-926b-1df9f64243b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:27.001469 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:43:27.056937 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ecb27a-f5fe-46d6-926b-1df9f64243b0 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:43:27.133339 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a05705a-7c47-44f0-9355-a2d177cee2db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:43:27.133956 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a05705a-7c47-44f0-9355-a2d177cee2db service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:27.134257 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a05705a-7c47-44f0-9355-a2d177cee2db service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:27.134508 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a05705a-7c47-44f0-9355-a2d177cee2db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:27.136230 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-3a05705a-7c47-44f0-9355-a2d177cee2db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:27.241002 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:43:27.259413 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a05705a-7c47-44f0-9355-a2d177cee2db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:43:27.260976 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a05705a-7c47-44f0-9355-a2d177cee2db service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:43:27.279520 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 670/1343] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:43:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:43:29.961362 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac991348-5328-4812-8d64-4570895ad2c4 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:29.962271 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac991348-5328-4812-8d64-4570895ad2c4 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:29.962271 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac991348-5328-4812-8d64-4570895ad2c4 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:29.963174 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 674/1344] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:43:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:43:29.968728 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ffa94-a0cc-4a8f-9f7e-171117174824 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:29.969246 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ffa94-a0cc-4a8f-9f7e-171117174824 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:29.969362 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ffa94-a0cc-4a8f-9f7e-171117174824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:30.042255 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-3c1ffa94-a0cc-4a8f-9f7e-171117174824 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:43:30.043469 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-3c1ffa94-a0cc-4a8f-9f7e-171117174824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:30.148672 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 671/1345] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:43:29 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:43:30.186641 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac991348-5328-4812-8d64-4570895ad2c4 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:43:30.188318 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-ac991348-5328-4812-8d64-4570895ad2c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:30.277153 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:43:30.295274 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac991348-5328-4812-8d64-4570895ad2c4 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:43:30.373063 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ea4987-5a1d-4b24-9773-42d1503c99bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:43:30.373671 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea4987-5a1d-4b24-9773-42d1503c99bd service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:30.373943 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea4987-5a1d-4b24-9773-42d1503c99bd service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:30.374210 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea4987-5a1d-4b24-9773-42d1503c99bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:30.375832 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-37ea4987-5a1d-4b24-9773-42d1503c99bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:30.489551 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ea4987-5a1d-4b24-9773-42d1503c99bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:43:30.490945 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ea4987-5a1d-4b24-9773-42d1503c99bd service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:43:30.509332 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 675/1346] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:43:30 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:43:34.471201 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ff83c-e7f6-4859-ba49-00b4dc43bfe0 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:34.471948 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ff83c-e7f6-4859-ba49-00b4dc43bfe0 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:34.471948 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ff83c-e7f6-4859-ba49-00b4dc43bfe0 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:34.473173 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 672/1347] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:43:34.478850 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41176dd7-7ac5-41cb-95dc-0f4b531be674 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:34.479325 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41176dd7-7ac5-41cb-95dc-0f4b531be674 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:34.479531 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41176dd7-7ac5-41cb-95dc-0f4b531be674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:34.495369 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:43:34.548436 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-41176dd7-7ac5-41cb-95dc-0f4b531be674 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:43:34.549676 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-41176dd7-7ac5-41cb-95dc-0f4b531be674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:34.672998 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 676/1348] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:43:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:43:34.712354 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276ff83c-e7f6-4859-ba49-00b4dc43bfe0 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:43:34.713887 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-276ff83c-e7f6-4859-ba49-00b4dc43bfe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:34.787298 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:43:34.840925 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276ff83c-e7f6-4859-ba49-00b4dc43bfe0 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:43:34.937433 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ef1f6-2f57-454d-b8e6-87403caad2bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:43:34.937832 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ef1f6-2f57-454d-b8e6-87403caad2bb service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:34.938113 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ef1f6-2f57-454d-b8e6-87403caad2bb service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:34.938387 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ef1f6-2f57-454d-b8e6-87403caad2bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:34.940132 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-260ef1f6-2f57-454d-b8e6-87403caad2bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:35.054674 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:43:35.059127 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260ef1f6-2f57-454d-b8e6-87403caad2bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:43:35.060660 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260ef1f6-2f57-454d-b8e6-87403caad2bb service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:43:35.080182 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 673/1349] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:43:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:43:40.105571 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920fbc4f-e81d-4581-a029-af05f25b015b None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:40.106424 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920fbc4f-e81d-4581-a029-af05f25b015b None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:40.106872 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920fbc4f-e81d-4581-a029-af05f25b015b None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:40.108707 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 677/1350] 10.222.0.20 () {54 vars in 921 bytes} [Tue Jul 26 12:43:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:43:40.114806 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f3415f-d428-41de-b2c0-27a4376f6f02 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:40.115568 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f3415f-d428-41de-b2c0-27a4376f6f02 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:40.116044 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f3415f-d428-41de-b2c0-27a4376f6f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:40.241911 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-f0f3415f-d428-41de-b2c0-27a4376f6f02 None None] MFA Rules not processed for user `ec49a03aed774c439459883bd5442d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:43:40.243199 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-f0f3415f-d428-41de-b2c0-27a4376f6f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:40.367610 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 674/1351] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Jul 26 12:43:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:43:44.505340 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0114db-6c5a-487f-845f-761d466313a8 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:44.505946 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0114db-6c5a-487f-845f-761d466313a8 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:44.506143 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0114db-6c5a-487f-845f-761d466313a8 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:44.508055 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 678/1352] 10.222.0.20 () {54 vars in 921 bytes} [Tue Jul 26 12:43:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:43:44.513299 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005547b1-9458-467b-b6ed-ebac1418f592 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:44.513768 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005547b1-9458-467b-b6ed-ebac1418f592 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:44.513857 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005547b1-9458-467b-b6ed-ebac1418f592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:44.573487 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:43:44.605184 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-005547b1-9458-467b-b6ed-ebac1418f592 None None] MFA Rules not processed for user `ec49a03aed774c439459883bd5442d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:43:44.606620 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-005547b1-9458-467b-b6ed-ebac1418f592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:44.671120 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=59 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:43:44.722599 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 675/1353] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Jul 26 12:43:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:43:46.036104 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483305b2-63c3-4f06-ae7d-ed7a2c0ebf92 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:46.036659 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483305b2-63c3-4f06-ae7d-ed7a2c0ebf92 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:46.036659 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483305b2-63c3-4f06-ae7d-ed7a2c0ebf92 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:46.038187 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 679/1354] 10.222.0.20 () {54 vars in 921 bytes} [Tue Jul 26 12:43:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:43:46.043330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d25091-ff6e-4d8d-93fa-bc017a4b28cd None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:46.043794 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d25091-ff6e-4d8d-93fa-bc017a4b28cd None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:46.043902 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d25091-ff6e-4d8d-93fa-bc017a4b28cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:46.121167 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-81d25091-ff6e-4d8d-93fa-bc017a4b28cd None None] MFA Rules not processed for user `ec49a03aed774c439459883bd5442d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:43:46.122620 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-81d25091-ff6e-4d8d-93fa-bc017a4b28cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:46.167325 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:43:46.238165 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 676/1355] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Jul 26 12:43:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:43:49.615150 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daecb44f-b72d-43a2-a9ab-65b04215cc2d None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:49.616040 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daecb44f-b72d-43a2-a9ab-65b04215cc2d None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:49.616491 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daecb44f-b72d-43a2-a9ab-65b04215cc2d None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:49.622172 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 680/1356] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:43:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:43:49.624782 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ccb12-b399-49d1-97ec-f300554373ce None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:49.625209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ccb12-b399-49d1-97ec-f300554373ce None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:49.625340 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ccb12-b399-49d1-97ec-f300554373ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:49.716980 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-9b7ccb12-b399-49d1-97ec-f300554373ce None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:43:49.718245 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-9b7ccb12-b399-49d1-97ec-f300554373ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:49.831778 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 677/1357] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:43:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:43:49.890881 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe3529d-c1d3-45fe-8312-99a14e933d86 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:49.891713 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe3529d-c1d3-45fe-8312-99a14e933d86 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:49.892266 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe3529d-c1d3-45fe-8312-99a14e933d86 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:49.893865 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 681/1358] 10.222.0.20 () {54 vars in 962 bytes} [Tue Jul 26 12:43:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:43:49.903515 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f10d305-1387-43ff-b5d0-7632f56243df None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:49.904517 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f10d305-1387-43ff-b5d0-7632f56243df None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:49.905912 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f10d305-1387-43ff-b5d0-7632f56243df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:49.927788 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:43:49.998988 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-4f10d305-1387-43ff-b5d0-7632f56243df None None] MFA Rules not processed for user `0f5d6f3c12244c0faf26048513dc49e7`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:43:50.000220 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-4f10d305-1387-43ff-b5d0-7632f56243df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:50.127102 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 678/1359] 10.222.0.20 () {58 vars in 1059 bytes} [Tue Jul 26 12:43:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:43:50.135532 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe3529d-c1d3-45fe-8312-99a14e933d86 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:43:50.137330 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-5fe3529d-c1d3-45fe-8312-99a14e933d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:50.297210 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe3529d-c1d3-45fe-8312-99a14e933d86 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:43:50.376701 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:43:50.421844 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5108268d-857a-43eb-9ee0-0eee2f5bbeae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:43:50.422466 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5108268d-857a-43eb-9ee0-0eee2f5bbeae service nova] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:50.422745 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5108268d-857a-43eb-9ee0-0eee2f5bbeae service nova] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:50.423018 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5108268d-857a-43eb-9ee0-0eee2f5bbeae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:50.425245 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-5108268d-857a-43eb-9ee0-0eee2f5bbeae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:50.519608 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5108268d-857a-43eb-9ee0-0eee2f5bbeae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:43:50.521235 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5108268d-857a-43eb-9ee0-0eee2f5bbeae service nova] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:43:50.541507 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 682/1360] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:43:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:43:52.804199 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d8832-bd43-48a5-94d1-c21550d27814 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:52.804199 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d8832-bd43-48a5-94d1-c21550d27814 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:52.804199 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d8832-bd43-48a5-94d1-c21550d27814 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:52.806814 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 679/1361] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:43:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:43:52.807254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f07b0-ff8f-4074-aa81-eb73f4c15797 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:52.807625 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f07b0-ff8f-4074-aa81-eb73f4c15797 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:52.808724 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f07b0-ff8f-4074-aa81-eb73f4c15797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:52.884424 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-ce8f07b0-ff8f-4074-aa81-eb73f4c15797 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:43:52.885643 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-ce8f07b0-ff8f-4074-aa81-eb73f4c15797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:52.929000 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:43:52.998485 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 683/1362] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:43:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:43:53.044031 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d8832-bd43-48a5-94d1-c21550d27814 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:43:53.045628 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-c48d8832-bd43-48a5-94d1-c21550d27814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:53.135482 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:43:53.239986 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d8832-bd43-48a5-94d1-c21550d27814 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:43:53.388142 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7ec4fd-4832-4967-b95d-5bb38ad38091 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:43:53.388766 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ec4fd-4832-4967-b95d-5bb38ad38091 service nova] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:53.389041 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ec4fd-4832-4967-b95d-5bb38ad38091 service nova] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:53.389309 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ec4fd-4832-4967-b95d-5bb38ad38091 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:53.390956 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-dd7ec4fd-4832-4967-b95d-5bb38ad38091 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:53.418755 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:43:53.511392 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7ec4fd-4832-4967-b95d-5bb38ad38091 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:43:53.513036 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7ec4fd-4832-4967-b95d-5bb38ad38091 service nova] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:43:53.533546 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 680/1363] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:43:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:43:55.935553 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076d44c-c53a-4e65-843e-fec5210c7f0d None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:55.936183 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076d44c-c53a-4e65-843e-fec5210c7f0d None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:55.936265 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076d44c-c53a-4e65-843e-fec5210c7f0d None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:55.937785 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 684/1364] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:43:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:43:55.943149 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572608b3-07d0-4740-894b-5b58fb8bacaa None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:55.943700 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572608b3-07d0-4740-894b-5b58fb8bacaa None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:55.943810 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572608b3-07d0-4740-894b-5b58fb8bacaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:56.028795 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-572608b3-07d0-4740-894b-5b58fb8bacaa None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:43:56.030435 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-572608b3-07d0-4740-894b-5b58fb8bacaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:56.132906 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:43:56.140685 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 681/1365] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:43:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:43:56.182281 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc85e6b-fb81-463a-8e5c-9ee0c263cd47 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:56.182568 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc85e6b-fb81-463a-8e5c-9ee0c263cd47 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:56.182842 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc85e6b-fb81-463a-8e5c-9ee0c263cd47 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:56.184434 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 685/1366] 10.222.0.20 () {54 vars in 962 bytes} [Tue Jul 26 12:43:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:43:56.190540 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae93cf7-a513-4cbf-be42-8c8599664d56 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:56.191182 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae93cf7-a513-4cbf-be42-8c8599664d56 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:56.191550 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae93cf7-a513-4cbf-be42-8c8599664d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:56.274111 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-9ae93cf7-a513-4cbf-be42-8c8599664d56 None None] MFA Rules not processed for user `0f5d6f3c12244c0faf26048513dc49e7`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:43:56.275330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-9ae93cf7-a513-4cbf-be42-8c8599664d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:56.411617 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 682/1367] 10.222.0.20 () {58 vars in 1059 bytes} [Tue Jul 26 12:43:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:43:56.420184 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc85e6b-fb81-463a-8e5c-9ee0c263cd47 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:43:56.421680 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-ebc85e6b-fb81-463a-8e5c-9ee0c263cd47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:56.598443 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc85e6b-fb81-463a-8e5c-9ee0c263cd47 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:43:56.620221 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:43:56.723456 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f95fc-cee7-44f8-bf43-93d808734197 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:43:56.724130 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f95fc-cee7-44f8-bf43-93d808734197 service nova] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:56.724413 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f95fc-cee7-44f8-bf43-93d808734197 service nova] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:56.724683 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f95fc-cee7-44f8-bf43-93d808734197 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:56.726296 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-db5f95fc-cee7-44f8-bf43-93d808734197 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:56.822127 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f95fc-cee7-44f8-bf43-93d808734197 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:43:56.824003 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f95fc-cee7-44f8-bf43-93d808734197 service nova] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:43:56.843470 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 686/1368] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:43:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:43:57.831912 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bf51d-b41a-42c8-97fc-843f30885c9b None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:57.832424 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bf51d-b41a-42c8-97fc-843f30885c9b None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:57.832424 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bf51d-b41a-42c8-97fc-843f30885c9b None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:57.833898 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 683/1369] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:43:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:43:57.838892 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa4b55-5548-439d-ac19-df1bfca74b3c None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:57.839306 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa4b55-5548-439d-ac19-df1bfca74b3c None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:57.839486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa4b55-5548-439d-ac19-df1bfca74b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:57.871833 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:43:57.918158 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-6faa4b55-5548-439d-ac19-df1bfca74b3c None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:43:57.919372 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-6faa4b55-5548-439d-ac19-df1bfca74b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:58.027748 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 687/1370] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:43:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:43:58.384520 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306602e4-b192-48bc-993d-f265fd5f4b36 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:58.384926 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306602e4-b192-48bc-993d-f265fd5f4b36 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:58.385309 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306602e4-b192-48bc-993d-f265fd5f4b36 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:58.387189 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 684/1371] 10.222.0.20 () {54 vars in 964 bytes} [Tue Jul 26 12:43:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:43:58.392206 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6c917-fa20-417a-b88a-4c8b2fd2c792 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:58.392602 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6c917-fa20-417a-b88a-4c8b2fd2c792 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:58.393646 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6c917-fa20-417a-b88a-4c8b2fd2c792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:58.475922 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-19d6c917-fa20-417a-b88a-4c8b2fd2c792 None None] MFA Rules not processed for user `9f8ca5e019dd4b9090cee2b2c1d8e15d`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:43:58.477408 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-19d6c917-fa20-417a-b88a-4c8b2fd2c792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:58.528517 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:43:58.567691 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 688/1372] 10.222.0.20 () {58 vars in 1061 bytes} [Tue Jul 26 12:43:58 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:43:58.572005 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306602e4-b192-48bc-993d-f265fd5f4b36 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:43:58.573622 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-306602e4-b192-48bc-993d-f265fd5f4b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:58.625716 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:43:58.687479 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306602e4-b192-48bc-993d-f265fd5f4b36 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:43:58.767859 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d11ed-3fbb-4a48-a898-ce44c7f4e60e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:43:58.768711 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d11ed-3fbb-4a48-a898-ce44c7f4e60e service cinder] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:58.769098 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d11ed-3fbb-4a48-a898-ce44c7f4e60e service cinder] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:58.769474 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d11ed-3fbb-4a48-a898-ce44c7f4e60e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:58.771202 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-092d11ed-3fbb-4a48-a898-ce44c7f4e60e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:58.888314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:43:58.907345 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092d11ed-3fbb-4a48-a898-ce44c7f4e60e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:43:58.909120 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092d11ed-3fbb-4a48-a898-ce44c7f4e60e service cinder] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:43:58.929946 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 685/1373] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:43:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:43:58.968198 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d45f07-d590-4759-989f-654842b94763 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:58.969330 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d45f07-d590-4759-989f-654842b94763 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:58.970050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d45f07-d590-4759-989f-654842b94763 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:58.973981 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 689/1374] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:43:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:43:58.978991 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72714baf-9a2f-42f5-ab34-0a286baafe4b None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:58.979258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72714baf-9a2f-42f5-ab34-0a286baafe4b None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:58.979686 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72714baf-9a2f-42f5-ab34-0a286baafe4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:59.048742 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-72714baf-9a2f-42f5-ab34-0a286baafe4b None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:43:59.050510 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-72714baf-9a2f-42f5-ab34-0a286baafe4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:59.155696 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 686/1375] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:43:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:43:59.205298 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d45f07-d590-4759-989f-654842b94763 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:43:59.206913 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-00d45f07-d590-4759-989f-654842b94763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:59.371450 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d45f07-d590-4759-989f-654842b94763 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:43:59.425908 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:43:59.490100 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b31e248-99c3-43c6-967e-6979c9f570f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:43:59.490758 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b31e248-99c3-43c6-967e-6979c9f570f2 service nova] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:43:59.491083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b31e248-99c3-43c6-967e-6979c9f570f2 service nova] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:43:59.491353 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b31e248-99c3-43c6-967e-6979c9f570f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:43:59.493013 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-3b31e248-99c3-43c6-967e-6979c9f570f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:43:59.582771 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b31e248-99c3-43c6-967e-6979c9f570f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:43:59.584475 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b31e248-99c3-43c6-967e-6979c9f570f2 service nova] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:43:59.603478 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 690/1376] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:43:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:44:01.651684 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b56b9-5cd6-480a-92a5-401d11fefee3 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:01.652220 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b56b9-5cd6-480a-92a5-401d11fefee3 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:01.652220 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b56b9-5cd6-480a-92a5-401d11fefee3 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:01.653602 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 687/1377] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:44:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:44:01.658779 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b82acf-4c89-4f70-aded-e08529dbeda3 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:01.659237 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b82acf-4c89-4f70-aded-e08529dbeda3 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:01.659371 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b82acf-4c89-4f70-aded-e08529dbeda3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:01.738788 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-d3b82acf-4c89-4f70-aded-e08529dbeda3 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:44:01.740053 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-d3b82acf-4c89-4f70-aded-e08529dbeda3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:01.747371 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:01.848100 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 691/1378] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:44:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:44:01.897537 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b56b9-5cd6-480a-92a5-401d11fefee3 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:44:01.899984 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-328b56b9-5cd6-480a-92a5-401d11fefee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:01.999525 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:02.078221 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b56b9-5cd6-480a-92a5-401d11fefee3 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:44:02.194919 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c34081-644d-4ad6-baa6-1fa62244e86d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:44:02.195560 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c34081-644d-4ad6-baa6-1fa62244e86d service nova] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:02.195864 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c34081-644d-4ad6-baa6-1fa62244e86d service nova] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:02.196133 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c34081-644d-4ad6-baa6-1fa62244e86d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:02.197752 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-33c34081-644d-4ad6-baa6-1fa62244e86d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:02.217434 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:02.290963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c34081-644d-4ad6-baa6-1fa62244e86d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:44:02.292605 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c34081-644d-4ad6-baa6-1fa62244e86d service nova] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:44:02.310314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 688/1379] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:44:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:44:04.433934 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538fc38-f0e3-4692-a572-d46700a864fc None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:04.434499 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538fc38-f0e3-4692-a572-d46700a864fc None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:04.434499 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538fc38-f0e3-4692-a572-d46700a864fc None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:04.435939 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 692/1380] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:44:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:44:04.441612 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5f86f-26cc-4788-9334-2b5d4288d8d7 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:04.442278 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5f86f-26cc-4788-9334-2b5d4288d8d7 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:04.442278 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5f86f-26cc-4788-9334-2b5d4288d8d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:04.521446 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-2ff5f86f-26cc-4788-9334-2b5d4288d8d7 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:44:04.522914 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-2ff5f86f-26cc-4788-9334-2b5d4288d8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:04.633876 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:04.643738 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 689/1381] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:44:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:44:04.686681 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538fc38-f0e3-4692-a572-d46700a864fc None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:44:04.688170 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-1538fc38-f0e3-4692-a572-d46700a864fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:04.850829 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:04.863885 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538fc38-f0e3-4692-a572-d46700a864fc None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:44:04.988320 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9852abb-5982-40f1-9c2c-9b951118bff1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:44:04.988944 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9852abb-5982-40f1-9c2c-9b951118bff1 service nova] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:04.989227 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9852abb-5982-40f1-9c2c-9b951118bff1 service nova] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:04.989503 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9852abb-5982-40f1-9c2c-9b951118bff1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:04.991288 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-c9852abb-5982-40f1-9c2c-9b951118bff1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:05.090774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9852abb-5982-40f1-9c2c-9b951118bff1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:44:05.092418 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9852abb-5982-40f1-9c2c-9b951118bff1 service nova] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:44:05.104438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:05.124064 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 693/1382] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:44:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:44:07.424902 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f885a-a4f0-40bb-88cd-cc0aed0fd4d3 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:07.425661 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f885a-a4f0-40bb-88cd-cc0aed0fd4d3 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:07.425661 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f885a-a4f0-40bb-88cd-cc0aed0fd4d3 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:07.426857 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 690/1383] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:44:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:44:07.432343 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c2799-0f22-4193-8129-e264eb324ab8 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:07.432825 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c2799-0f22-4193-8129-e264eb324ab8 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:07.432825 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c2799-0f22-4193-8129-e264eb324ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:07.500816 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-b16c2799-0f22-4193-8129-e264eb324ab8 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:44:07.502031 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-b16c2799-0f22-4193-8129-e264eb324ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:07.609287 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 694/1384] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:44:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:44:07.653216 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2f885a-a4f0-40bb-88cd-cc0aed0fd4d3 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:44:07.654853 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-4e2f885a-a4f0-40bb-88cd-cc0aed0fd4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:07.807080 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2f885a-a4f0-40bb-88cd-cc0aed0fd4d3 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:44:07.878437 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:07.930427 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504c57d0-c046-4043-9e04-5a5837e349ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:44:07.931039 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c57d0-c046-4043-9e04-5a5837e349ae service nova] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:07.931316 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c57d0-c046-4043-9e04-5a5837e349ae service nova] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:07.931592 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c57d0-c046-4043-9e04-5a5837e349ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:07.933249 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-504c57d0-c046-4043-9e04-5a5837e349ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:08.024690 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504c57d0-c046-4043-9e04-5a5837e349ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:44:08.026825 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504c57d0-c046-4043-9e04-5a5837e349ae service nova] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:44:08.046850 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 691/1385] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:44:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:44:10.385715 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a2b45-b1ad-4b84-ae03-67409be4ca3a None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:10.386262 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a2b45-b1ad-4b84-ae03-67409be4ca3a None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:10.386262 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a2b45-b1ad-4b84-ae03-67409be4ca3a None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:10.387588 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 695/1386] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:44:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:44:10.393561 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a962a-161a-4e95-a7e9-b0405ad0f24c None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:10.394013 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a962a-161a-4e95-a7e9-b0405ad0f24c None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:10.394352 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a962a-161a-4e95-a7e9-b0405ad0f24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:10.475363 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-340a962a-161a-4e95-a7e9-b0405ad0f24c None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:44:10.476604 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-340a962a-161a-4e95-a7e9-b0405ad0f24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:10.511330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:10.592270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 692/1387] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:44:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:44:10.650537 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a2b45-b1ad-4b84-ae03-67409be4ca3a None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:44:10.652098 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-366a2b45-b1ad-4b84-ae03-67409be4ca3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:10.733326 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:10.828336 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a2b45-b1ad-4b84-ae03-67409be4ca3a None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:44:10.965792 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfa7f77-f68e-4dd8-8d99-3635266382c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:44:10.966546 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa7f77-f68e-4dd8-8d99-3635266382c5 service nova] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:10.966907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa7f77-f68e-4dd8-8d99-3635266382c5 service nova] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:10.967255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa7f77-f68e-4dd8-8d99-3635266382c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:10.969477 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-8cfa7f77-f68e-4dd8-8d99-3635266382c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:11.013764 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:11.073929 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfa7f77-f68e-4dd8-8d99-3635266382c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:44:11.076024 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfa7f77-f68e-4dd8-8d99-3635266382c5 service nova] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:44:11.101241 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 696/1388] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:44:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:44:13.359756 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910440e4-37f5-4a0a-9eff-d7290ed43936 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:13.360325 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910440e4-37f5-4a0a-9eff-d7290ed43936 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:13.360325 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910440e4-37f5-4a0a-9eff-d7290ed43936 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:13.361891 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 693/1389] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:44:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:44:13.367080 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccd335-06a5-4f17-8479-2861866f241d None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:13.367512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccd335-06a5-4f17-8479-2861866f241d None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:13.367617 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccd335-06a5-4f17-8479-2861866f241d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:13.432540 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-ebccd335-06a5-4f17-8479-2861866f241d None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:44:13.433649 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-ebccd335-06a5-4f17-8479-2861866f241d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:13.542193 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 697/1390] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:44:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:44:13.588104 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910440e4-37f5-4a0a-9eff-d7290ed43936 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:44:13.589625 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-910440e4-37f5-4a0a-9eff-d7290ed43936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:13.746004 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910440e4-37f5-4a0a-9eff-d7290ed43936 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:44:13.763423 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:13.866296 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a65fc-fc76-443e-9d14-22fd1e1d1616 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:44:13.866906 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a65fc-fc76-443e-9d14-22fd1e1d1616 service nova] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:13.867180 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a65fc-fc76-443e-9d14-22fd1e1d1616 service nova] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:13.867448 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a65fc-fc76-443e-9d14-22fd1e1d1616 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:13.869104 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-9d1a65fc-fc76-443e-9d14-22fd1e1d1616 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:13.960381 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1a65fc-fc76-443e-9d14-22fd1e1d1616 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:44:13.962028 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1a65fc-fc76-443e-9d14-22fd1e1d1616 service nova] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:44:13.980649 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 694/1391] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:44:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:44:14.264812 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10361a6d-f401-4c4d-842b-1eef92535471 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:14.265059 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10361a6d-f401-4c4d-842b-1eef92535471 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:14.265311 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10361a6d-f401-4c4d-842b-1eef92535471 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:14.266678 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 698/1392] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:44:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:44:14.272773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c96f5-a30e-4aa6-83d7-5e2fd1401686 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:14.273419 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c96f5-a30e-4aa6-83d7-5e2fd1401686 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:14.273419 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c96f5-a30e-4aa6-83d7-5e2fd1401686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:14.300037 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:14.343992 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-542c96f5-a30e-4aa6-83d7-5e2fd1401686 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:44:14.345192 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-542c96f5-a30e-4aa6-83d7-5e2fd1401686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:14.441794 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 695/1393] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:44:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:44:14.496854 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220bc7cf-c47f-4f24-9f89-21edf2f51b41 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:14.497325 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220bc7cf-c47f-4f24-9f89-21edf2f51b41 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:14.497430 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220bc7cf-c47f-4f24-9f89-21edf2f51b41 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:14.499014 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 699/1394] 10.222.0.20 () {54 vars in 966 bytes} [Tue Jul 26 12:44:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:44:14.504919 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97690ca2-9727-4367-ad0e-0167559a114d None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:14.505210 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97690ca2-9727-4367-ad0e-0167559a114d None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:14.505485 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97690ca2-9727-4367-ad0e-0167559a114d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:14.582757 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-97690ca2-9727-4367-ad0e-0167559a114d None None] MFA Rules not processed for user `d2b5853838dc40db89c46038c0606107`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:44:14.583922 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-97690ca2-9727-4367-ad0e-0167559a114d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:14.638953 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:14.676024 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 696/1395] 10.222.0.20 () {58 vars in 1063 bytes} [Tue Jul 26 12:44:14 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:44:14.683760 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220bc7cf-c47f-4f24-9f89-21edf2f51b41 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:44:14.685272 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-220bc7cf-c47f-4f24-9f89-21edf2f51b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:14.699628 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:14.795544 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220bc7cf-c47f-4f24-9f89-21edf2f51b41 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:44:14.871462 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a8b561-57d7-4e49-ae99-66735246bd35 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2b5853838dc40db89c46038c0606107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:44:14.872047 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8b561-57d7-4e49-ae99-66735246bd35 service glance] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:14.872322 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8b561-57d7-4e49-ae99-66735246bd35 service glance] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:14.872596 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8b561-57d7-4e49-ae99-66735246bd35 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:14.874480 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-28a8b561-57d7-4e49-ae99-66735246bd35 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:14.943311 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:14.985980 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a8b561-57d7-4e49-ae99-66735246bd35 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:44:14.987467 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a8b561-57d7-4e49-ae99-66735246bd35 service glance] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:44:15.006466 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 700/1396] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:44:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:44:15.748718 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352aba6-7856-4f81-9868-42e15030dc44 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:15.749229 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352aba6-7856-4f81-9868-42e15030dc44 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:15.749336 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352aba6-7856-4f81-9868-42e15030dc44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:15.818983 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-3352aba6-7856-4f81-9868-42e15030dc44 None None] MFA Rules not processed for user `30fc532f464843a3ad35e4d3c387bb3a`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:44:15.820213 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-3352aba6-7856-4f81-9868-42e15030dc44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:15.912763 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 697/1397] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Jul 26 12:44:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:44:15.958188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ec261-8db5-4002-8825-845af13e58ed None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:15.958900 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ec261-8db5-4002-8825-845af13e58ed None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:15.958900 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ec261-8db5-4002-8825-845af13e58ed None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:15.960124 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 701/1398] 10.222.0.20 () {54 vars in 943 bytes} [Tue Jul 26 12:44:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:44:15.966494 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b644ddd-9779-4e9a-975a-f0ee1ca7094e None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:15.966804 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b644ddd-9779-4e9a-975a-f0ee1ca7094e None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:15.967095 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b644ddd-9779-4e9a-975a-f0ee1ca7094e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:16.016562 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e78155-0d64-4958-aaf6-9b1a136704ae None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:16.016769 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e78155-0d64-4958-aaf6-9b1a136704ae None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:16.017014 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e78155-0d64-4958-aaf6-9b1a136704ae None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:16.018401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 702/1399] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:44:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 12:44:16.023513 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64c609-439d-4574-b096-501ac5e3a867 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:16.023831 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64c609-439d-4574-b096-501ac5e3a867 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:16.024111 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64c609-439d-4574-b096-501ac5e3a867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:16.039625 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:16.047878 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-2b644ddd-9779-4e9a-975a-f0ee1ca7094e None None] MFA Rules not processed for user `cd2483f74d7d48b1a7b8ffe32f63abde`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:44:16.049616 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-2b644ddd-9779-4e9a-975a-f0ee1ca7094e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:16.093805 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-8a64c609-439d-4574-b096-501ac5e3a867 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:44:16.095027 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-8a64c609-439d-4574-b096-501ac5e3a867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:16.157155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 698/1400] 10.222.0.20 () {58 vars in 1040 bytes} [Tue Jul 26 12:44:15 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:44:16.167239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b644ddd-9779-4e9a-975a-f0ee1ca7094e None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:44:16.177639 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b644ddd-9779-4e9a-975a-f0ee1ca7094e None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:44:16.200271 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 703/1401] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:44:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:44:16.246280 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a64c609-439d-4574-b096-501ac5e3a867 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:44:16.247475 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-8a64c609-439d-4574-b096-501ac5e3a867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:16.274707 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:16.288549 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f28b90-e64f-4689-88cc-2e4386ff0077 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd2483f74d7d48b1a7b8ffe32f63abde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:44:16.289182 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f28b90-e64f-4689-88cc-2e4386ff0077 service swift] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:16.289460 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f28b90-e64f-4689-88cc-2e4386ff0077 service swift] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:16.289734 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f28b90-e64f-4689-88cc-2e4386ff0077 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:16.291392 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-29f28b90-e64f-4689-88cc-2e4386ff0077 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:16.293981 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:16.366265 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a64c609-439d-4574-b096-501ac5e3a867 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:44:16.379579 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f28b90-e64f-4689-88cc-2e4386ff0077 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:44:16.381229 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f28b90-e64f-4689-88cc-2e4386ff0077 service swift] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:44:16.389374 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 699/1402] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Jul 26 12:44:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 26 12:44:16.494178 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1693e0a9-1785-4984-a88b-51541b72bd42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:44:16.494804 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693e0a9-1785-4984-a88b-51541b72bd42 service nova] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:16.495080 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693e0a9-1785-4984-a88b-51541b72bd42 service nova] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:16.495358 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693e0a9-1785-4984-a88b-51541b72bd42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:16.497015 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-1693e0a9-1785-4984-a88b-51541b72bd42 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:16.519975 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:16.601791 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1693e0a9-1785-4984-a88b-51541b72bd42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:44:16.603810 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1693e0a9-1785-4984-a88b-51541b72bd42 service nova] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:44:16.623537 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 704/1403] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:44:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 12:44:18.706051 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49253b-ed81-4a40-8ca8-301428ea359f None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:18.706667 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49253b-ed81-4a40-8ca8-301428ea359f None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:18.706667 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49253b-ed81-4a40-8ca8-301428ea359f None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:18.708257 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 700/1404] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:44:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:44:18.713575 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a9b024-7692-4227-a1dd-a6e4105ad97c None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:18.714024 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a9b024-7692-4227-a1dd-a6e4105ad97c None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:18.714132 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a9b024-7692-4227-a1dd-a6e4105ad97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:18.786523 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-71a9b024-7692-4227-a1dd-a6e4105ad97c None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:44:18.787768 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-71a9b024-7692-4227-a1dd-a6e4105ad97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:18.888411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:18.896202 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 705/1405] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:44:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:44:18.941561 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b49253b-ed81-4a40-8ca8-301428ea359f None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:44:18.946528 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-8b49253b-ed81-4a40-8ca8-301428ea359f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:19.111609 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:19.114507 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b49253b-ed81-4a40-8ca8-301428ea359f None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:44:19.242332 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8725f493-de2e-433f-8337-55e6587be737 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:44:19.243262 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8725f493-de2e-433f-8337-55e6587be737 service nova] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:19.243537 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8725f493-de2e-433f-8337-55e6587be737 service nova] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:19.243836 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8725f493-de2e-433f-8337-55e6587be737 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:19.245460 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-8725f493-de2e-433f-8337-55e6587be737 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:19.358262 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8725f493-de2e-433f-8337-55e6587be737 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:44:19.359939 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8725f493-de2e-433f-8337-55e6587be737 service nova] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:44:19.396390 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 701/1406] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:44:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:44:21.478665 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99120e-39d7-4538-8626-9fe6a3cbae95 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:21.479230 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99120e-39d7-4538-8626-9fe6a3cbae95 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:21.479230 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99120e-39d7-4538-8626-9fe6a3cbae95 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:21.480833 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 706/1407] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:44:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:44:21.486702 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d14d8-f11f-4170-b8e3-3be3a2e0df74 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:21.487148 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d14d8-f11f-4170-b8e3-3be3a2e0df74 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:21.487262 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d14d8-f11f-4170-b8e3-3be3a2e0df74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:21.492366 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:21.560629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-308d14d8-f11f-4170-b8e3-3be3a2e0df74 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:44:21.562067 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-308d14d8-f11f-4170-b8e3-3be3a2e0df74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:21.781626 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 702/1408] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:44:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:44:21.820379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a99120e-39d7-4538-8626-9fe6a3cbae95 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:44:21.821909 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-3a99120e-39d7-4538-8626-9fe6a3cbae95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:21.987630 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a99120e-39d7-4538-8626-9fe6a3cbae95 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:44:22.065763 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:22.116841 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763be0d6-a0b6-4f96-9444-42d4f4034595 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:44:22.117584 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763be0d6-a0b6-4f96-9444-42d4f4034595 service nova] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:22.117930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763be0d6-a0b6-4f96-9444-42d4f4034595 service nova] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:22.118269 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763be0d6-a0b6-4f96-9444-42d4f4034595 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:22.120006 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-763be0d6-a0b6-4f96-9444-42d4f4034595 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:22.218503 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763be0d6-a0b6-4f96-9444-42d4f4034595 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:44:22.220683 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763be0d6-a0b6-4f96-9444-42d4f4034595 service nova] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:44:22.239731 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 707/1409] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:44:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:44:31.771166 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=64 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:32.237420 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=64 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:39.283533 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a8c05-e522-4ba7-8445-20340beaed40 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:39.283993 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a8c05-e522-4ba7-8445-20340beaed40 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:39.284097 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a8c05-e522-4ba7-8445-20340beaed40 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:39.285492 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 703/1410] 10.222.0.20 () {54 vars in 918 bytes} [Tue Jul 26 12:44:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:44:39.290521 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846518c-6d83-414a-8662-ce988fc4a90b None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:39.290957 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846518c-6d83-414a-8662-ce988fc4a90b None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:39.291068 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846518c-6d83-414a-8662-ce988fc4a90b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:39.366205 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-5846518c-6d83-414a-8662-ce988fc4a90b None None] MFA Rules not processed for user `ec49a03aed774c439459883bd5442d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:44:39.367402 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-5846518c-6d83-414a-8662-ce988fc4a90b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:39.477801 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 708/1411] 10.222.0.20 () {58 vars in 1015 bytes} [Tue Jul 26 12:44:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:44:39.493536 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4352b-7a3c-46ca-a371-cf2b5f6fae7a None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:39.493922 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4352b-7a3c-46ca-a371-cf2b5f6fae7a None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:39.494430 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4352b-7a3c-46ca-a371-cf2b5f6fae7a None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:39.496191 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 704/1412] 10.222.0.20 () {54 vars in 918 bytes} [Tue Jul 26 12:44:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:44:39.501289 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf51cbfb-27f0-4096-99c9-e20202f51783 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:39.501581 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf51cbfb-27f0-4096-99c9-e20202f51783 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:39.501862 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf51cbfb-27f0-4096-99c9-e20202f51783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:39.573834 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:39.578481 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-cf51cbfb-27f0-4096-99c9-e20202f51783 None None] MFA Rules not processed for user `0f5d6f3c12244c0faf26048513dc49e7`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:44:39.580321 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-cf51cbfb-27f0-4096-99c9-e20202f51783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:39.694739 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 709/1413] 10.222.0.20 () {58 vars in 1015 bytes} [Tue Jul 26 12:44:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:44:42.218109 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a7bdc-6cce-4806-9451-1c578d7fd4ee None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:42.218639 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a7bdc-6cce-4806-9451-1c578d7fd4ee None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:42.218639 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a7bdc-6cce-4806-9451-1c578d7fd4ee None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:42.220094 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 705/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 12:44:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:44:42.225394 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f5fab-fe33-4170-86f9-24193e6151b6 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:42.225828 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f5fab-fe33-4170-86f9-24193e6151b6 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:42.225949 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f5fab-fe33-4170-86f9-24193e6151b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:42.291608 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-488f5fab-fe33-4170-86f9-24193e6151b6 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:44:42.292852 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-488f5fab-fe33-4170-86f9-24193e6151b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:42.349116 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:42.393329 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 710/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 12:44:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:44:42.436018 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a7bdc-6cce-4806-9451-1c578d7fd4ee None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:44:42.437478 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-215a7bdc-6cce-4806-9451-1c578d7fd4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:42.580316 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a7bdc-6cce-4806-9451-1c578d7fd4ee None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:44:42.651435 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:42.692951 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00c4dc6-b1d6-4bf2-836f-6fa52c173d1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:44:42.693512 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c4dc6-b1d6-4bf2-836f-6fa52c173d1a service nova] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:42.693792 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c4dc6-b1d6-4bf2-836f-6fa52c173d1a service nova] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:42.694066 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c4dc6-b1d6-4bf2-836f-6fa52c173d1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:42.695789 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-a00c4dc6-b1d6-4bf2-836f-6fa52c173d1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:42.783878 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00c4dc6-b1d6-4bf2-836f-6fa52c173d1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:44:42.785519 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00c4dc6-b1d6-4bf2-836f-6fa52c173d1a service nova] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:44:42.803831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 706/1416] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:44:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:44:43.724058 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a3ff9-b16f-41f0-9da5-39e556a8ca16 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:43.724639 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a3ff9-b16f-41f0-9da5-39e556a8ca16 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:43.724639 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a3ff9-b16f-41f0-9da5-39e556a8ca16 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:43.725951 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 711/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 12:44:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:44:43.730768 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624cc7a-03ab-4b59-99f5-939dd424e8df None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:43.731171 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624cc7a-03ab-4b59-99f5-939dd424e8df None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:43.731316 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624cc7a-03ab-4b59-99f5-939dd424e8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:43.807220 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-c624cc7a-03ab-4b59-99f5-939dd424e8df None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:44:43.808454 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-c624cc7a-03ab-4b59-99f5-939dd424e8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:43.835308 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:44.028544 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 707/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 12:44:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:44:44.183219 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a3ff9-b16f-41f0-9da5-39e556a8ca16 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:44:44.184126 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-eb5a3ff9-b16f-41f0-9da5-39e556a8ca16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:44.329144 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a3ff9-b16f-41f0-9da5-39e556a8ca16 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:44:44.369132 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:44.444454 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f73948-c1e1-4a2c-8675-006fe4d7db2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:44:44.445070 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f73948-c1e1-4a2c-8675-006fe4d7db2b service nova] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:44.445354 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f73948-c1e1-4a2c-8675-006fe4d7db2b service nova] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:44.445630 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f73948-c1e1-4a2c-8675-006fe4d7db2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:44.447254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-73f73948-c1e1-4a2c-8675-006fe4d7db2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:44.537123 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f73948-c1e1-4a2c-8675-006fe4d7db2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:44:44.538719 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f73948-c1e1-4a2c-8675-006fe4d7db2b service nova] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:44:44.556802 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 712/1419] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:44:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 12:44:45.444168 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248d36a-1296-4e41-b7d2-4cd96cb48d4e None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:45.444682 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248d36a-1296-4e41-b7d2-4cd96cb48d4e None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:45.444682 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248d36a-1296-4e41-b7d2-4cd96cb48d4e None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:45.446000 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 708/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 12:44:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:44:45.450826 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd9e58-85ac-4cdc-bed5-786f07226ed9 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:45.451217 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd9e58-85ac-4cdc-bed5-786f07226ed9 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:45.451381 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd9e58-85ac-4cdc-bed5-786f07226ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:45.507241 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:45.529538 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-b6bd9e58-85ac-4cdc-bed5-786f07226ed9 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:44:45.530771 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-b6bd9e58-85ac-4cdc-bed5-786f07226ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:45.626598 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 713/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 12:44:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:44:45.777049 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248d36a-1296-4e41-b7d2-4cd96cb48d4e None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:44:45.778713 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-8248d36a-1296-4e41-b7d2-4cd96cb48d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:45.942251 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248d36a-1296-4e41-b7d2-4cd96cb48d4e None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:44:45.955610 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:46.062097 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fdf365-a8c0-4058-8f30-7320f5534a82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:44:46.062653 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fdf365-a8c0-4058-8f30-7320f5534a82 service nova] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:46.062933 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fdf365-a8c0-4058-8f30-7320f5534a82 service nova] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:46.063202 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fdf365-a8c0-4058-8f30-7320f5534a82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:46.064862 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-35fdf365-a8c0-4058-8f30-7320f5534a82 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:46.152449 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fdf365-a8c0-4058-8f30-7320f5534a82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:44:46.154036 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fdf365-a8c0-4058-8f30-7320f5534a82 service nova] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:44:46.172398 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 709/1422] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:44:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:44:47.042249 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737376f-04ac-417e-ba24-06200f8c09b0 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:47.042800 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737376f-04ac-417e-ba24-06200f8c09b0 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:47.042800 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737376f-04ac-417e-ba24-06200f8c09b0 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:47.044194 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 714/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 12:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:44:47.049932 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df812e8-0385-4770-b2f4-54a37e97ae76 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:47.050361 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df812e8-0385-4770-b2f4-54a37e97ae76 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:47.050496 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df812e8-0385-4770-b2f4-54a37e97ae76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:47.070092 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:47.127222 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-1df812e8-0385-4770-b2f4-54a37e97ae76 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:44:47.128448 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-1df812e8-0385-4770-b2f4-54a37e97ae76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:47.227049 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 710/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 12:44:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:44:47.378239 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e737376f-04ac-417e-ba24-06200f8c09b0 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:44:47.379732 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-e737376f-04ac-417e-ba24-06200f8c09b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:47.491225 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:47.527214 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e737376f-04ac-417e-ba24-06200f8c09b0 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:44:47.641763 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573b2a4c-bbda-42c0-a6ed-8879dbc53b80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:44:47.642296 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b2a4c-bbda-42c0-a6ed-8879dbc53b80 service nova] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:47.642578 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b2a4c-bbda-42c0-a6ed-8879dbc53b80 service nova] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:47.642862 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b2a4c-bbda-42c0-a6ed-8879dbc53b80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:47.644521 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-573b2a4c-bbda-42c0-a6ed-8879dbc53b80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:47.721706 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:47.735418 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573b2a4c-bbda-42c0-a6ed-8879dbc53b80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:44:47.737072 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573b2a4c-bbda-42c0-a6ed-8879dbc53b80 service nova] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:44:47.755774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 715/1425] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:44:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:44:48.624309 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0200b9cc-c2f9-4836-aa34-51c8bbe25cc4 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:48.625406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0200b9cc-c2f9-4836-aa34-51c8bbe25cc4 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:48.625406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0200b9cc-c2f9-4836-aa34-51c8bbe25cc4 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:48.626747 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 711/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 12:44:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:44:48.631771 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcbb6a-9dd8-48b5-9cf3-a16ef5e1cc94 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:48.632229 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcbb6a-9dd8-48b5-9cf3-a16ef5e1cc94 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:48.632336 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcbb6a-9dd8-48b5-9cf3-a16ef5e1cc94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:48.707291 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-c6fcbb6a-9dd8-48b5-9cf3-a16ef5e1cc94 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:44:48.708528 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-c6fcbb6a-9dd8-48b5-9cf3-a16ef5e1cc94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:48.803194 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 716/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 12:44:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:44:48.960103 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0200b9cc-c2f9-4836-aa34-51c8bbe25cc4 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:44:48.961652 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-0200b9cc-c2f9-4836-aa34-51c8bbe25cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:49.044295 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:49.106653 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0200b9cc-c2f9-4836-aa34-51c8bbe25cc4 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:44:49.222043 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbce4dd2-7783-43e7-b9e3-7ad584e8dbdc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:44:49.222583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce4dd2-7783-43e7-b9e3-7ad584e8dbdc service nova] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:49.222866 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce4dd2-7783-43e7-b9e3-7ad584e8dbdc service nova] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:49.223139 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce4dd2-7783-43e7-b9e3-7ad584e8dbdc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:49.224831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-fbce4dd2-7783-43e7-b9e3-7ad584e8dbdc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:49.274174 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:49.316044 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbce4dd2-7783-43e7-b9e3-7ad584e8dbdc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:44:49.317642 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbce4dd2-7783-43e7-b9e3-7ad584e8dbdc service nova] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:44:49.335770 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 712/1428] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:44:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:44:50.137572 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd963bb-f253-450b-8d39-a2bfe0d7d698 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:50.138148 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd963bb-f253-450b-8d39-a2bfe0d7d698 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:50.138148 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd963bb-f253-450b-8d39-a2bfe0d7d698 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:50.139493 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 717/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 12:44:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:44:50.144682 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5024d03b-5da4-4141-adb4-a155c2a48feb None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:50.145102 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5024d03b-5da4-4141-adb4-a155c2a48feb None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:50.145234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5024d03b-5da4-4141-adb4-a155c2a48feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:50.222121 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-5024d03b-5da4-4141-adb4-a155c2a48feb None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:44:50.223326 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-5024d03b-5da4-4141-adb4-a155c2a48feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:50.320017 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 713/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 12:44:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:44:50.470887 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd963bb-f253-450b-8d39-a2bfe0d7d698 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:44:50.472437 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-7dd963bb-f253-450b-8d39-a2bfe0d7d698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:50.521058 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:50.618777 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd963bb-f253-450b-8d39-a2bfe0d7d698 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:44:50.732418 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705f53c5-e03b-4ec5-b04c-af375a2070e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:44:50.732943 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f53c5-e03b-4ec5-b04c-af375a2070e0 service nova] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:50.733219 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f53c5-e03b-4ec5-b04c-af375a2070e0 service nova] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:50.733486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f53c5-e03b-4ec5-b04c-af375a2070e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:50.735073 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-705f53c5-e03b-4ec5-b04c-af375a2070e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:50.741936 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:50.826843 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705f53c5-e03b-4ec5-b04c-af375a2070e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:44:50.829151 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705f53c5-e03b-4ec5-b04c-af375a2070e0 service nova] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:44:50.846980 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 718/1431] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:44:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:44:51.677484 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be3a4c1-d674-43a6-b4fe-df12d918110b None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:51.678048 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be3a4c1-d674-43a6-b4fe-df12d918110b None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:51.678048 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be3a4c1-d674-43a6-b4fe-df12d918110b None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:51.679392 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 714/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 12:44:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:44:51.684441 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8747b-26f7-452d-a867-641033239261 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:51.684924 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8747b-26f7-452d-a867-641033239261 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:51.685037 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8747b-26f7-452d-a867-641033239261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:51.767576 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-c4c8747b-26f7-452d-a867-641033239261 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:44:51.768848 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-c4c8747b-26f7-452d-a867-641033239261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:51.836299 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:51.865624 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 719/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 12:44:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:44:52.029259 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be3a4c1-d674-43a6-b4fe-df12d918110b None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:44:52.030872 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-5be3a4c1-d674-43a6-b4fe-df12d918110b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:52.048698 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:52.175593 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be3a4c1-d674-43a6-b4fe-df12d918110b None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:44:52.262922 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:52.290531 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef017e-ae54-4c15-8710-d5e825ab8a73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:44:52.291298 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef017e-ae54-4c15-8710-d5e825ab8a73 service nova] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:52.291694 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef017e-ae54-4c15-8710-d5e825ab8a73 service nova] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:52.292078 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef017e-ae54-4c15-8710-d5e825ab8a73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:52.293854 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-47ef017e-ae54-4c15-8710-d5e825ab8a73 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:52.392016 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef017e-ae54-4c15-8710-d5e825ab8a73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:44:52.393601 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef017e-ae54-4c15-8710-d5e825ab8a73 service nova] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:44:52.412059 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 715/1434] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:44:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:44:53.228615 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cebc1ea-2982-4a11-bfa7-a909ff0b7f26 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:53.229195 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cebc1ea-2982-4a11-bfa7-a909ff0b7f26 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:53.229195 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cebc1ea-2982-4a11-bfa7-a909ff0b7f26 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:53.230504 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 720/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 12:44:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:44:53.235455 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b095a68-b838-44e5-b095-2e2ab00fdfd8 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:53.235933 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b095a68-b838-44e5-b095-2e2ab00fdfd8 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:53.236061 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b095a68-b838-44e5-b095-2e2ab00fdfd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:53.317387 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-2b095a68-b838-44e5-b095-2e2ab00fdfd8 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:44:53.318512 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-2b095a68-b838-44e5-b095-2e2ab00fdfd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:53.360675 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:53.415025 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 716/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 12:44:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:44:53.565894 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cebc1ea-2982-4a11-bfa7-a909ff0b7f26 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:44:53.567464 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-5cebc1ea-2982-4a11-bfa7-a909ff0b7f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:53.715013 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cebc1ea-2982-4a11-bfa7-a909ff0b7f26 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:44:53.769372 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:53.829907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70bea74-93f0-42a4-8cf3-c3ca809869e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:44:53.830445 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70bea74-93f0-42a4-8cf3-c3ca809869e4 service nova] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:53.830723 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70bea74-93f0-42a4-8cf3-c3ca809869e4 service nova] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:53.831003 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70bea74-93f0-42a4-8cf3-c3ca809869e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:53.832671 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-f70bea74-93f0-42a4-8cf3-c3ca809869e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:53.929301 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70bea74-93f0-42a4-8cf3-c3ca809869e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:44:53.931317 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70bea74-93f0-42a4-8cf3-c3ca809869e4 service nova] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:44:53.949877 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 721/1437] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:44:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:44:54.757059 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874a30b-fb7d-440f-bfcc-8d99edc71267 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:54.757620 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874a30b-fb7d-440f-bfcc-8d99edc71267 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:54.757620 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874a30b-fb7d-440f-bfcc-8d99edc71267 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:54.759118 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 717/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 12:44:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:44:54.764095 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a027d4-a3b6-4d2d-8488-16a27fa88bc1 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:54.764529 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a027d4-a3b6-4d2d-8488-16a27fa88bc1 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:54.764636 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a027d4-a3b6-4d2d-8488-16a27fa88bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:54.845365 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-94a027d4-a3b6-4d2d-8488-16a27fa88bc1 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:44:54.846582 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-94a027d4-a3b6-4d2d-8488-16a27fa88bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:54.848652 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:54.944174 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 722/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 12:44:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:44:55.096797 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6874a30b-fb7d-440f-bfcc-8d99edc71267 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:44:55.098301 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-6874a30b-fb7d-440f-bfcc-8d99edc71267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:55.242274 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6874a30b-fb7d-440f-bfcc-8d99edc71267 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:44:55.275069 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:55.364179 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ec2ad-ac9f-4f26-998d-2fa7b1519bd0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:44:55.364795 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ec2ad-ac9f-4f26-998d-2fa7b1519bd0 service nova] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:55.365076 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ec2ad-ac9f-4f26-998d-2fa7b1519bd0 service nova] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:55.365345 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ec2ad-ac9f-4f26-998d-2fa7b1519bd0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:55.366966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-9d5ec2ad-ac9f-4f26-998d-2fa7b1519bd0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:55.454795 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5ec2ad-ac9f-4f26-998d-2fa7b1519bd0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:44:55.456443 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5ec2ad-ac9f-4f26-998d-2fa7b1519bd0 service nova] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:44:55.474389 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 718/1440] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:44:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:44:57.427731 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638f2f9-19ff-4128-8dae-78d9a34f0ea2 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:57.428292 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638f2f9-19ff-4128-8dae-78d9a34f0ea2 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:57.428292 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638f2f9-19ff-4128-8dae-78d9a34f0ea2 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:57.429758 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 723/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 12:44:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:44:57.435064 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f349df63-dde9-41be-8406-71f8e0600370 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:57.435514 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f349df63-dde9-41be-8406-71f8e0600370 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:57.435650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f349df63-dde9-41be-8406-71f8e0600370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:57.477070 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:57.503129 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-f349df63-dde9-41be-8406-71f8e0600370 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:44:57.504385 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-f349df63-dde9-41be-8406-71f8e0600370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:57.603843 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 719/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 12:44:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:44:57.643785 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2638f2f9-19ff-4128-8dae-78d9a34f0ea2 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:44:57.645338 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-2638f2f9-19ff-4128-8dae-78d9a34f0ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:57.750203 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2638f2f9-19ff-4128-8dae-78d9a34f0ea2 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:44:57.782242 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:44:57.824446 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208fe7df-9420-43e4-a814-f39c8d66a4c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:44:57.825062 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208fe7df-9420-43e4-a814-f39c8d66a4c9 service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:44:57.825351 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208fe7df-9420-43e4-a814-f39c8d66a4c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:44:57.825624 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208fe7df-9420-43e4-a814-f39c8d66a4c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:44:57.827242 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-208fe7df-9420-43e4-a814-f39c8d66a4c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:44:57.939803 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208fe7df-9420-43e4-a814-f39c8d66a4c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:44:57.941261 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208fe7df-9420-43e4-a814-f39c8d66a4c9 service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:44:57.960454 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 724/1443] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:44:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:45:00.122205 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274cfded-6a0b-46a9-8ef3-ef327e94b13c None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:00.122737 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274cfded-6a0b-46a9-8ef3-ef327e94b13c None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:00.122737 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274cfded-6a0b-46a9-8ef3-ef327e94b13c None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:00.124690 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 720/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 12:45:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:45:00.130139 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b7365-f7e3-444e-a9fb-470b6689edc5 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:00.130566 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b7365-f7e3-444e-a9fb-470b6689edc5 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:00.130669 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b7365-f7e3-444e-a9fb-470b6689edc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:00.196430 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-d16b7365-f7e3-444e-a9fb-470b6689edc5 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:45:00.197645 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-d16b7365-f7e3-444e-a9fb-470b6689edc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:00.210351 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:45:00.299130 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 725/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 12:45:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:45:00.337733 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274cfded-6a0b-46a9-8ef3-ef327e94b13c None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:45:00.339290 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-274cfded-6a0b-46a9-8ef3-ef327e94b13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:00.442995 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274cfded-6a0b-46a9-8ef3-ef327e94b13c None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:45:00.454792 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:45:00.515352 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946c7766-8780-4c08-8ff0-9c233aeef2ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:45:00.515983 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c7766-8780-4c08-8ff0-9c233aeef2ea service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:00.516261 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c7766-8780-4c08-8ff0-9c233aeef2ea service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:00.516527 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c7766-8780-4c08-8ff0-9c233aeef2ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:00.518142 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-946c7766-8780-4c08-8ff0-9c233aeef2ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:00.626282 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946c7766-8780-4c08-8ff0-9c233aeef2ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:45:00.627752 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946c7766-8780-4c08-8ff0-9c233aeef2ea service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:45:00.645548 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 721/1446] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:45:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:45:03.018519 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be64f0-4ff6-4dc1-a292-d46d5754a1d6 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:03.019143 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be64f0-4ff6-4dc1-a292-d46d5754a1d6 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:03.019143 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be64f0-4ff6-4dc1-a292-d46d5754a1d6 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:03.020481 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 726/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 12:45:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:45:03.025566 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523269b-9173-4330-a177-7071da08ede3 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:03.025970 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523269b-9173-4330-a177-7071da08ede3 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:03.026115 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523269b-9173-4330-a177-7071da08ede3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:03.075179 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:45:03.094412 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-4523269b-9173-4330-a177-7071da08ede3 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:45:03.095623 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-4523269b-9173-4330-a177-7071da08ede3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:03.196955 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 722/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 12:45:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:45:03.235367 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56be64f0-4ff6-4dc1-a292-d46d5754a1d6 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:45:03.236992 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-56be64f0-4ff6-4dc1-a292-d46d5754a1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:03.341096 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56be64f0-4ff6-4dc1-a292-d46d5754a1d6 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:45:03.383206 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:45:03.417682 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b4aa07-5335-4bec-9b36-886409e2dd5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:45:03.418298 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b4aa07-5335-4bec-9b36-886409e2dd5a service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:03.418579 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b4aa07-5335-4bec-9b36-886409e2dd5a service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:03.418853 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b4aa07-5335-4bec-9b36-886409e2dd5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:03.420590 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-c0b4aa07-5335-4bec-9b36-886409e2dd5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:03.528367 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b4aa07-5335-4bec-9b36-886409e2dd5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:45:03.529796 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b4aa07-5335-4bec-9b36-886409e2dd5a service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:45:03.548871 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 727/1449] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:45:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:45:05.687618 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9dfbf-cd86-46d4-a8a0-eefd54cfb7f0 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:05.688232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9dfbf-cd86-46d4-a8a0-eefd54cfb7f0 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:05.688334 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9dfbf-cd86-46d4-a8a0-eefd54cfb7f0 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:05.689692 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 723/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 12:45:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:45:05.695005 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce0673-9cef-457f-b92e-894358b3c02e None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:05.695475 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce0673-9cef-457f-b92e-894358b3c02e None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:05.695673 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce0673-9cef-457f-b92e-894358b3c02e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:05.763104 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-8bce0673-9cef-457f-b92e-894358b3c02e None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:45:05.764348 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-8bce0673-9cef-457f-b92e-894358b3c02e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:05.785265 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:45:05.865453 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 728/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 12:45:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:45:05.904808 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd9dfbf-cd86-46d4-a8a0-eefd54cfb7f0 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:45:05.906890 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-3dd9dfbf-cd86-46d4-a8a0-eefd54cfb7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:06.010926 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd9dfbf-cd86-46d4-a8a0-eefd54cfb7f0 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:45:06.031621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:45:06.092170 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188588f7-fce7-43fb-8520-e125689dda98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:45:06.093384 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188588f7-fce7-43fb-8520-e125689dda98 service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:06.093666 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188588f7-fce7-43fb-8520-e125689dda98 service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:06.093935 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188588f7-fce7-43fb-8520-e125689dda98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:06.095557 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-188588f7-fce7-43fb-8520-e125689dda98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:06.205325 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188588f7-fce7-43fb-8520-e125689dda98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:45:06.206766 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188588f7-fce7-43fb-8520-e125689dda98 service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:45:06.225197 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 724/1452] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:45:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 12:45:08.391085 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ecbb51-ef20-465c-b778-2fa82bb514f0 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:08.391673 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ecbb51-ef20-465c-b778-2fa82bb514f0 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:08.391673 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ecbb51-ef20-465c-b778-2fa82bb514f0 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:08.393011 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 729/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 12:45:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:45:08.398459 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c87d1-a8e9-48d5-ad06-4cd38f834aae None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:08.398917 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c87d1-a8e9-48d5-ad06-4cd38f834aae None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:08.399017 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c87d1-a8e9-48d5-ad06-4cd38f834aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:08.462161 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:45:08.464754 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-c84c87d1-a8e9-48d5-ad06-4cd38f834aae None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:45:08.466547 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-c84c87d1-a8e9-48d5-ad06-4cd38f834aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:08.562172 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 725/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 12:45:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:45:08.599797 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ecbb51-ef20-465c-b778-2fa82bb514f0 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:45:08.601324 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-18ecbb51-ef20-465c-b778-2fa82bb514f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:08.726284 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ecbb51-ef20-465c-b778-2fa82bb514f0 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:45:08.775500 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:45:08.801250 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d26bd5-4577-46c1-a566-3c6c1cd7b16e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:45:08.801862 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d26bd5-4577-46c1-a566-3c6c1cd7b16e service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:08.802147 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d26bd5-4577-46c1-a566-3c6c1cd7b16e service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:08.802422 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d26bd5-4577-46c1-a566-3c6c1cd7b16e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:08.804090 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-96d26bd5-4577-46c1-a566-3c6c1cd7b16e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:08.917069 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d26bd5-4577-46c1-a566-3c6c1cd7b16e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:45:08.918527 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d26bd5-4577-46c1-a566-3c6c1cd7b16e service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:45:08.936997 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 730/1455] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:45:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:45:11.082846 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb369e6-58e0-46b9-9796-4ed3c351c9be None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:11.083426 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb369e6-58e0-46b9-9796-4ed3c351c9be None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:11.083426 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb369e6-58e0-46b9-9796-4ed3c351c9be None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:11.084857 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 726/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 12:45:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:45:11.090058 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8be92-b4cd-4a8c-9f4b-0fbe314524ce None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:11.090506 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8be92-b4cd-4a8c-9f4b-0fbe314524ce None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:11.090612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8be92-b4cd-4a8c-9f4b-0fbe314524ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:11.165117 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-4ca8be92-b4cd-4a8c-9f4b-0fbe314524ce None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:45:11.166392 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-4ca8be92-b4cd-4a8c-9f4b-0fbe314524ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:11.197964 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:45:11.266878 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 731/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 12:45:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:45:11.304533 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb369e6-58e0-46b9-9796-4ed3c351c9be None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:45:11.306400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-4eb369e6-58e0-46b9-9796-4ed3c351c9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:11.420148 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb369e6-58e0-46b9-9796-4ed3c351c9be None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:45:11.450565 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:45:11.497542 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8497ba42-4844-4a70-9f72-ac147f70e5bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:45:11.498160 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497ba42-4844-4a70-9f72-ac147f70e5bb service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:11.498439 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497ba42-4844-4a70-9f72-ac147f70e5bb service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:11.499334 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497ba42-4844-4a70-9f72-ac147f70e5bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:11.501181 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-8497ba42-4844-4a70-9f72-ac147f70e5bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:11.614402 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8497ba42-4844-4a70-9f72-ac147f70e5bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:45:11.616005 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8497ba42-4844-4a70-9f72-ac147f70e5bb service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:45:11.637087 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 727/1458] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:45:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:45:13.711835 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0839f-df5d-41bf-9e4d-9e93c8c2f6e2 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:13.712328 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0839f-df5d-41bf-9e4d-9e93c8c2f6e2 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:13.712436 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0839f-df5d-41bf-9e4d-9e93c8c2f6e2 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:13.713755 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 732/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 12:45:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:45:13.719021 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c0f01-ad59-46d7-983c-79d333842ed0 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:13.719514 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c0f01-ad59-46d7-983c-79d333842ed0 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:13.719621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c0f01-ad59-46d7-983c-79d333842ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:13.784789 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-8e9c0f01-ad59-46d7-983c-79d333842ed0 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:45:13.786000 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-8e9c0f01-ad59-46d7-983c-79d333842ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:13.794428 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:45:13.889004 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 728/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 12:45:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:45:13.926963 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0839f-df5d-41bf-9e4d-9e93c8c2f6e2 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:45:13.928562 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-9bb0839f-df5d-41bf-9e4d-9e93c8c2f6e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:14.042600 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0839f-df5d-41bf-9e4d-9e93c8c2f6e2 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:45:14.099088 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:45:14.118194 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f2d61c-359b-4f5d-8525-4cc01b94b311 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:45:14.118809 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f2d61c-359b-4f5d-8525-4cc01b94b311 service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:14.119091 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f2d61c-359b-4f5d-8525-4cc01b94b311 service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:14.119365 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f2d61c-359b-4f5d-8525-4cc01b94b311 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:14.121016 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-54f2d61c-359b-4f5d-8525-4cc01b94b311 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:14.231683 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f2d61c-359b-4f5d-8525-4cc01b94b311 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:45:14.233735 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f2d61c-359b-4f5d-8525-4cc01b94b311 service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:45:14.252378 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 733/1461] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:45:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:45:16.299399 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82002b22-4463-4b88-81cb-33c82570e3fb None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:16.299969 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82002b22-4463-4b88-81cb-33c82570e3fb None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:16.299969 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82002b22-4463-4b88-81cb-33c82570e3fb None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:16.301313 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 729/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 12:45:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:45:16.306653 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98d9aa-caf6-48d7-88cf-92dc299cdc3d None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:16.307063 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98d9aa-caf6-48d7-88cf-92dc299cdc3d None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:16.307257 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98d9aa-caf6-48d7-88cf-92dc299cdc3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:16.372936 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-eb98d9aa-caf6-48d7-88cf-92dc299cdc3d None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:45:16.374625 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-eb98d9aa-caf6-48d7-88cf-92dc299cdc3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:16.413151 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:45:16.472170 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 734/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 12:45:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:45:16.509995 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82002b22-4463-4b88-81cb-33c82570e3fb None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:45:16.511554 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-82002b22-4463-4b88-81cb-33c82570e3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:16.614454 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82002b22-4463-4b88-81cb-33c82570e3fb None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:45:16.652156 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:45:16.688457 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d8092c-bafb-4264-9140-24b5320f36c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:45:16.689071 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d8092c-bafb-4264-9140-24b5320f36c1 service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:16.689351 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d8092c-bafb-4264-9140-24b5320f36c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:16.689623 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d8092c-bafb-4264-9140-24b5320f36c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:16.691253 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-c3d8092c-bafb-4264-9140-24b5320f36c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:16.802042 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d8092c-bafb-4264-9140-24b5320f36c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:45:16.803394 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d8092c-bafb-4264-9140-24b5320f36c1 service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:45:16.821231 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 730/1464] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:45:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:45:18.841911 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd4175-91ac-47e9-9a95-c28199ceb423 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:18.842526 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd4175-91ac-47e9-9a95-c28199ceb423 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:18.842526 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd4175-91ac-47e9-9a95-c28199ceb423 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:18.843907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 735/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 12:45:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:45:18.849106 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ab49b-5f35-488c-b50d-aa51186a66f3 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:18.849560 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ab49b-5f35-488c-b50d-aa51186a66f3 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:18.849664 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ab49b-5f35-488c-b50d-aa51186a66f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:18.916681 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-484ab49b-5f35-488c-b50d-aa51186a66f3 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:45:18.917884 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-484ab49b-5f35-488c-b50d-aa51186a66f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:18.936761 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:45:19.015338 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 731/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 12:45:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:45:19.051811 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdd4175-91ac-47e9-9a95-c28199ceb423 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:45:19.053364 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-efdd4175-91ac-47e9-9a95-c28199ceb423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:19.155760 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdd4175-91ac-47e9-9a95-c28199ceb423 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:45:19.226668 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:45:19.234090 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e3f08b-9788-4021-a6cf-88ce4674e79a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:45:19.234702 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e3f08b-9788-4021-a6cf-88ce4674e79a service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:19.234979 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e3f08b-9788-4021-a6cf-88ce4674e79a service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:19.235245 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e3f08b-9788-4021-a6cf-88ce4674e79a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:19.236878 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-30e3f08b-9788-4021-a6cf-88ce4674e79a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:19.347550 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e3f08b-9788-4021-a6cf-88ce4674e79a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:45:19.349036 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e3f08b-9788-4021-a6cf-88ce4674e79a service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:45:19.367614 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 736/1467] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:45:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:45:21.408118 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742dce3-1189-437c-99a5-164ba0b7ec10 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:21.408692 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742dce3-1189-437c-99a5-164ba0b7ec10 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:21.408692 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742dce3-1189-437c-99a5-164ba0b7ec10 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:21.410048 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 732/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 12:45:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:45:21.415236 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b437c6ba-87b6-4fab-a81f-a761a1a55c2e None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:21.415694 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b437c6ba-87b6-4fab-a81f-a761a1a55c2e None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:21.415844 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b437c6ba-87b6-4fab-a81f-a761a1a55c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:21.498903 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-b437c6ba-87b6-4fab-a81f-a761a1a55c2e None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:45:21.500178 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-b437c6ba-87b6-4fab-a81f-a761a1a55c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:21.548912 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:45:21.597456 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 737/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 12:45:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:45:21.635947 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f742dce3-1189-437c-99a5-164ba0b7ec10 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:45:21.637493 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-f742dce3-1189-437c-99a5-164ba0b7ec10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:21.740414 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f742dce3-1189-437c-99a5-164ba0b7ec10 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:45:21.792547 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:45:21.820453 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4646ad-f628-4bd4-a5fc-b52751350d99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:45:21.821063 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4646ad-f628-4bd4-a5fc-b52751350d99 service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:21.821341 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4646ad-f628-4bd4-a5fc-b52751350d99 service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:21.821609 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4646ad-f628-4bd4-a5fc-b52751350d99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:21.823216 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-2b4646ad-f628-4bd4-a5fc-b52751350d99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:21.932869 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4646ad-f628-4bd4-a5fc-b52751350d99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:45:21.934295 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4646ad-f628-4bd4-a5fc-b52751350d99 service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:45:21.952394 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 733/1470] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:45:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:45:31.595555 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=22 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:45:31.949662 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=66 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:45:41.554747 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6fe41b-a6b7-48ca-b071-43ffa0edcd1b None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:41.555322 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6fe41b-a6b7-48ca-b071-43ffa0edcd1b None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:41.555322 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6fe41b-a6b7-48ca-b071-43ffa0edcd1b None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:41.556693 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 738/1471] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:45:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:45:41.564845 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9d9bd-a32e-49e2-b38e-6a9236488612 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:41.564845 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9d9bd-a32e-49e2-b38e-6a9236488612 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:41.564845 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9d9bd-a32e-49e2-b38e-6a9236488612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:41.644240 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-c1a9d9bd-a32e-49e2-b38e-6a9236488612 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:45:41.645447 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-c1a9d9bd-a32e-49e2-b38e-6a9236488612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:41.741787 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 734/1472] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:45:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:45:41.827474 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf009b3-dcb7-41ac-9304-ef541ee4f226 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:41.827806 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf009b3-dcb7-41ac-9304-ef541ee4f226 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:41.828089 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf009b3-dcb7-41ac-9304-ef541ee4f226 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:41.829459 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 739/1473] 10.222.0.20 () {54 vars in 966 bytes} [Tue Jul 26 12:45:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:45:41.834313 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ff7f2-4a95-4401-b18a-974c4b75b2d0 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:41.834597 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ff7f2-4a95-4401-b18a-974c4b75b2d0 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:41.834868 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ff7f2-4a95-4401-b18a-974c4b75b2d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:41.922586 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-239ff7f2-4a95-4401-b18a-974c4b75b2d0 None None] MFA Rules not processed for user `d2b5853838dc40db89c46038c0606107`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:45:41.923832 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-239ff7f2-4a95-4401-b18a-974c4b75b2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:41.947440 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:45:42.006097 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 735/1474] 10.222.0.20 () {58 vars in 1063 bytes} [Tue Jul 26 12:45:41 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:45:42.013710 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf009b3-dcb7-41ac-9304-ef541ee4f226 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:45:42.015249 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-0bf009b3-dcb7-41ac-9304-ef541ee4f226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:42.117792 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf009b3-dcb7-41ac-9304-ef541ee4f226 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:45:42.191297 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989d9d75-c9ab-435c-8ec1-1fed06a41949 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2b5853838dc40db89c46038c0606107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:45:42.191941 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d9d75-c9ab-435c-8ec1-1fed06a41949 service glance] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:42.192219 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d9d75-c9ab-435c-8ec1-1fed06a41949 service glance] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:42.192493 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d9d75-c9ab-435c-8ec1-1fed06a41949 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:42.194095 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-989d9d75-c9ab-435c-8ec1-1fed06a41949 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:42.374418 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:45:42.429779 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989d9d75-c9ab-435c-8ec1-1fed06a41949 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:45:42.431224 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989d9d75-c9ab-435c-8ec1-1fed06a41949 service glance] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:45:42.449201 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 740/1475] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:45:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:45:44.547036 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d918c9-39a4-48ff-8430-dd2cdca56872 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:44.547625 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d918c9-39a4-48ff-8430-dd2cdca56872 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:44.547625 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d918c9-39a4-48ff-8430-dd2cdca56872 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:44.548965 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 736/1476] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:45:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:45:44.554104 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f09fa97-7e39-4249-99f5-80fb3b1e7d34 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:44.554530 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f09fa97-7e39-4249-99f5-80fb3b1e7d34 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:44.554644 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f09fa97-7e39-4249-99f5-80fb3b1e7d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:44.627572 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-5f09fa97-7e39-4249-99f5-80fb3b1e7d34 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:45:44.628835 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-5f09fa97-7e39-4249-99f5-80fb3b1e7d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:44.724000 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 741/1477] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:45:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:45:44.747498 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d918c9-39a4-48ff-8430-dd2cdca56872 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:45:44.749097 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-d6d918c9-39a4-48ff-8430-dd2cdca56872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:44.874196 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d918c9-39a4-48ff-8430-dd2cdca56872 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:45:44.923433 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:45:44.971831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4695141a-3e07-4926-acad-b8abea25f937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:45:44.972440 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695141a-3e07-4926-acad-b8abea25f937 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:44.972716 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695141a-3e07-4926-acad-b8abea25f937 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:44.972982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695141a-3e07-4926-acad-b8abea25f937 admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:44.973663 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4695141a-3e07-4926-acad-b8abea25f937 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:45:44.975359 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: WARNING py.warnings [None req-4695141a-3e07-4926-acad-b8abea25f937 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 12:45:44.975359 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: warnings.warn(msg) Jul 26 12:45:44.975359 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: Jul 26 12:45:44.976515 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4695141a-3e07-4926-acad-b8abea25f937 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:45:44.994840 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 737/1478] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 26 12:45:44 2022] GET /identity/v3/projects => generated 2553 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:45:47.034919 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe5725-f7f6-4a93-ad02-fa04dadfe558 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:47.035499 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe5725-f7f6-4a93-ad02-fa04dadfe558 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:47.035499 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe5725-f7f6-4a93-ad02-fa04dadfe558 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:47.036865 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 742/1479] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:45:47.042114 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e9185-ccbb-40b8-bcdf-1752a16b0e84 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:47.042532 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e9185-ccbb-40b8-bcdf-1752a16b0e84 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:47.042705 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e9185-ccbb-40b8-bcdf-1752a16b0e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:47.108272 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-3c3e9185-ccbb-40b8-bcdf-1752a16b0e84 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:45:47.109497 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-3c3e9185-ccbb-40b8-bcdf-1752a16b0e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:47.207400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 738/1480] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:45:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:45:47.250792 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abe5725-f7f6-4a93-ad02-fa04dadfe558 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:45:47.252317 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-5abe5725-f7f6-4a93-ad02-fa04dadfe558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:47.262057 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:45:47.396320 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abe5725-f7f6-4a93-ad02-fa04dadfe558 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:45:47.478801 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:45:47.511659 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5887ba-6b8b-42b5-8954-53bb20c1dea9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:45:47.512264 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5887ba-6b8b-42b5-8954-53bb20c1dea9 service nova] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:47.513068 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5887ba-6b8b-42b5-8954-53bb20c1dea9 service nova] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:47.513339 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5887ba-6b8b-42b5-8954-53bb20c1dea9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:47.514961 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-2e5887ba-6b8b-42b5-8954-53bb20c1dea9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:47.603167 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5887ba-6b8b-42b5-8954-53bb20c1dea9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:45:47.604702 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5887ba-6b8b-42b5-8954-53bb20c1dea9 service nova] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:45:47.622598 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 743/1481] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:45:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:45:49.811990 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe75063-9e02-4bb2-a88d-57bba6b7ef19 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:49.812646 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe75063-9e02-4bb2-a88d-57bba6b7ef19 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:49.812646 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe75063-9e02-4bb2-a88d-57bba6b7ef19 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:49.813807 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 739/1482] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:45:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 12:45:49.818970 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caeb924-db40-4cf6-8e81-b1f783553c7b None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:49.819403 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caeb924-db40-4cf6-8e81-b1f783553c7b None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:49.819519 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caeb924-db40-4cf6-8e81-b1f783553c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:49.888325 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-4caeb924-db40-4cf6-8e81-b1f783553c7b None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:45:49.889538 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-4caeb924-db40-4cf6-8e81-b1f783553c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:49.935862 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:45:49.987178 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 744/1483] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:45:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:45:50.022658 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe75063-9e02-4bb2-a88d-57bba6b7ef19 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:45:50.024231 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-ebe75063-9e02-4bb2-a88d-57bba6b7ef19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:50.048770 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:45:50.129621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe75063-9e02-4bb2-a88d-57bba6b7ef19 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:45:50.202754 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b8bf9e-6d5f-4a5c-a660-58ba1f1c6343 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:45:50.203382 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8bf9e-6d5f-4a5c-a660-58ba1f1c6343 service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:50.203690 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8bf9e-6d5f-4a5c-a660-58ba1f1c6343 service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:50.203970 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8bf9e-6d5f-4a5c-a660-58ba1f1c6343 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:50.205607 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-85b8bf9e-6d5f-4a5c-a660-58ba1f1c6343 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:50.280183 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:45:50.316082 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b8bf9e-6d5f-4a5c-a660-58ba1f1c6343 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:45:50.317523 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b8bf9e-6d5f-4a5c-a660-58ba1f1c6343 service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:45:50.337889 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 740/1484] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:45:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:45:52.398858 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba79b66-f3c3-4e9a-b6bf-7a6403eb6e05 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:52.399425 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba79b66-f3c3-4e9a-b6bf-7a6403eb6e05 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:52.399425 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba79b66-f3c3-4e9a-b6bf-7a6403eb6e05 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:52.400799 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 745/1485] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:45:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:45:52.406486 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3078e-209d-43f9-b8b9-8e13177938e8 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:52.406970 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3078e-209d-43f9-b8b9-8e13177938e8 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:52.407137 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3078e-209d-43f9-b8b9-8e13177938e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:52.474577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-5ca3078e-209d-43f9-b8b9-8e13177938e8 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:45:52.475833 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-5ca3078e-209d-43f9-b8b9-8e13177938e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:52.576419 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 741/1486] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:45:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:45:52.614521 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba79b66-f3c3-4e9a-b6bf-7a6403eb6e05 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:45:52.616107 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-fba79b66-f3c3-4e9a-b6bf-7a6403eb6e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:52.719359 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba79b66-f3c3-4e9a-b6bf-7a6403eb6e05 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:45:52.791614 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:45:52.793640 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52173575-5f4e-49c2-ae79-d9f257357755 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:45:52.794290 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52173575-5f4e-49c2-ae79-d9f257357755 service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:52.794765 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52173575-5f4e-49c2-ae79-d9f257357755 service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:52.795281 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52173575-5f4e-49c2-ae79-d9f257357755 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:52.797135 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-52173575-5f4e-49c2-ae79-d9f257357755 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:52.911403 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52173575-5f4e-49c2-ae79-d9f257357755 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:45:52.912887 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52173575-5f4e-49c2-ae79-d9f257357755 service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:45:52.930680 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 746/1487] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:45:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:45:55.138505 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d5668-9166-4c5b-a79a-22b577d6a008 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:55.139095 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d5668-9166-4c5b-a79a-22b577d6a008 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:55.139095 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d5668-9166-4c5b-a79a-22b577d6a008 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:55.140500 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 742/1488] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:45:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:45:55.145985 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c459e9d1-16ae-4a53-8683-cd306358e689 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:55.146424 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c459e9d1-16ae-4a53-8683-cd306358e689 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:55.146532 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c459e9d1-16ae-4a53-8683-cd306358e689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:55.213682 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-c459e9d1-16ae-4a53-8683-cd306358e689 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:45:55.214901 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-c459e9d1-16ae-4a53-8683-cd306358e689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:55.267470 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:45:55.312126 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 747/1489] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:45:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:45:55.354309 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d5668-9166-4c5b-a79a-22b577d6a008 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:45:55.357122 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-d26d5668-9166-4c5b-a79a-22b577d6a008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:55.393945 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:45:55.461573 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d5668-9166-4c5b-a79a-22b577d6a008 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:45:55.537355 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b2cbb-b77f-4dd6-b79a-ddc7e157d6fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:45:55.537979 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b2cbb-b77f-4dd6-b79a-ddc7e157d6fe service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:55.538264 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b2cbb-b77f-4dd6-b79a-ddc7e157d6fe service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:55.538541 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b2cbb-b77f-4dd6-b79a-ddc7e157d6fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:55.540228 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-1b0b2cbb-b77f-4dd6-b79a-ddc7e157d6fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:55.623507 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:45:55.653426 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0b2cbb-b77f-4dd6-b79a-ddc7e157d6fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:45:55.654790 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0b2cbb-b77f-4dd6-b79a-ddc7e157d6fe service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:45:55.672739 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 743/1490] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:45:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:45:57.926997 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0405b-5f7a-4263-a855-6107a33afff0 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:57.927538 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0405b-5f7a-4263-a855-6107a33afff0 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:57.927538 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0405b-5f7a-4263-a855-6107a33afff0 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:57.928851 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 748/1491] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 26 12:45:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:45:57.934153 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd3e82-fa47-478d-994b-049ae00f99e8 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:57.934567 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd3e82-fa47-478d-994b-049ae00f99e8 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:57.934708 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd3e82-fa47-478d-994b-049ae00f99e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:58.002362 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-f2cd3e82-fa47-478d-994b-049ae00f99e8 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:45:58.003581 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-f2cd3e82-fa47-478d-994b-049ae00f99e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:58.107932 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 744/1492] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 26 12:45:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:45:58.148524 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e0405b-5f7a-4263-a855-6107a33afff0 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:45:58.150088 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-95e0405b-5f7a-4263-a855-6107a33afff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:58.253609 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e0405b-5f7a-4263-a855-6107a33afff0 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:45:58.326556 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08257a5f-2a1e-4bf2-928c-0b543deaae25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:45:58.327173 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08257a5f-2a1e-4bf2-928c-0b543deaae25 service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:45:58.327448 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08257a5f-2a1e-4bf2-928c-0b543deaae25 service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:45:58.327763 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08257a5f-2a1e-4bf2-928c-0b543deaae25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:45:58.329381 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-08257a5f-2a1e-4bf2-928c-0b543deaae25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:45:58.338561 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:45:58.444325 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08257a5f-2a1e-4bf2-928c-0b543deaae25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:45:58.445675 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08257a5f-2a1e-4bf2-928c-0b543deaae25 service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:45:58.463551 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 749/1493] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:45:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 12:46:08.106041 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=82 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:46:08.461263 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=50 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:47:31.174429 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34596386-5c0a-42bb-a018-05c18b8f6432 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:47:31.175026 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34596386-5c0a-42bb-a018-05c18b8f6432 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:47:31.175026 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34596386-5c0a-42bb-a018-05c18b8f6432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:47:31.249262 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-34596386-5c0a-42bb-a018-05c18b8f6432 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:47:31.250454 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-34596386-5c0a-42bb-a018-05c18b8f6432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:47:31.344338 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 745/1494] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:47:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:47:31.352635 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3775b2d-2b46-4d2c-9479-91a446372804 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:47:31.353122 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3775b2d-2b46-4d2c-9479-91a446372804 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:47:31.353230 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3775b2d-2b46-4d2c-9479-91a446372804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:47:31.430475 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-d3775b2d-2b46-4d2c-9479-91a446372804 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:47:31.431606 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-d3775b2d-2b46-4d2c-9479-91a446372804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:47:31.529471 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 750/1495] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:47:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:47:31.534162 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34596386-5c0a-42bb-a018-05c18b8f6432 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:47:31.535405 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-34596386-5c0a-42bb-a018-05c18b8f6432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:47:31.547556 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34596386-5c0a-42bb-a018-05c18b8f6432 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:47:31.609372 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:47:31.644955 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c9c0d0-fad3-4e6f-935e-da808be37d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:47:31.645879 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9c0d0-fad3-4e6f-935e-da808be37d41 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:47:31.646162 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9c0d0-fad3-4e6f-935e-da808be37d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:47:31.646528 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9c0d0-fad3-4e6f-935e-da808be37d41 admin admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:47:31.647298 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c9c0d0-fad3-4e6f-935e-da808be37d41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:47:31.647595 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c9c0d0-fad3-4e6f-935e-da808be37d41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:47:31.649271 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: WARNING py.warnings [None req-e6c9c0d0-fad3-4e6f-935e-da808be37d41 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 26 12:47:31.649271 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: warnings.warn(msg) Jul 26 12:47:31.649271 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: Jul 26 12:47:31.650350 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c9c0d0-fad3-4e6f-935e-da808be37d41 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:47:31.662547 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 746/1496] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:47:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:47:31.670970 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3775b2d-2b46-4d2c-9479-91a446372804 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:47:31.680269 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3775b2d-2b46-4d2c-9479-91a446372804 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:47:31.741154 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:47:31.775947 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79315316-9c4b-49c5-9046-f7dcec73e05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:47:31.776560 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79315316-9c4b-49c5-9046-f7dcec73e05f admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:47:31.776839 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79315316-9c4b-49c5-9046-f7dcec73e05f admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:47:31.777110 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79315316-9c4b-49c5-9046-f7dcec73e05f admin admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:47:31.777782 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79315316-9c4b-49c5-9046-f7dcec73e05f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:47:31.779512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: WARNING py.warnings [None req-79315316-9c4b-49c5-9046-f7dcec73e05f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 12:47:31.779512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: warnings.warn(msg) Jul 26 12:47:31.779512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: Jul 26 12:47:31.780767 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79315316-9c4b-49c5-9046-f7dcec73e05f admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:47:31.838223 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 751/1497] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:47:31 2022] POST /identity/v3/projects => generated 379 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:47:31.842484 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c9c0d0-fad3-4e6f-935e-da808be37d41 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:47:31.844114 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-e6c9c0d0-fad3-4e6f-935e-da808be37d41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:47:31.877188 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c9c0d0-fad3-4e6f-935e-da808be37d41 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:47:31.970878 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5bd0d2-a133-40e1-89a1-40709fd90aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:47:31.971630 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5bd0d2-a133-40e1-89a1-40709fd90aca admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:47:31.971950 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5bd0d2-a133-40e1-89a1-40709fd90aca admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:47:31.972221 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5bd0d2-a133-40e1-89a1-40709fd90aca admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:47:31.973024 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5bd0d2-a133-40e1-89a1-40709fd90aca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:47:31.974028 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: WARNING py.warnings [None req-da5bd0d2-a133-40e1-89a1-40709fd90aca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 12:47:31.974028 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: warnings.warn(msg) Jul 26 12:47:31.974028 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: Jul 26 12:47:31.975229 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5bd0d2-a133-40e1-89a1-40709fd90aca admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:47:32.018374 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 747/1498] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:47:31 2022] POST /identity/v3/users => generated 290 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:47:32.022448 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79315316-9c4b-49c5-9046-f7dcec73e05f admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:47:32.023983 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-79315316-9c4b-49c5-9046-f7dcec73e05f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:47:32.087462 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79315316-9c4b-49c5-9046-f7dcec73e05f admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:47:32.170772 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:47:32.184718 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daa677d-9c98-4fe0-9ac2-33c7f2232f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:47:32.185337 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa677d-9c98-4fe0-9ac2-33c7f2232f30 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:47:32.185611 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa677d-9c98-4fe0-9ac2-33c7f2232f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:47:32.185875 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa677d-9c98-4fe0-9ac2-33c7f2232f30 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:47:32.186566 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daa677d-9c98-4fe0-9ac2-33c7f2232f30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:47:32.187598 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: WARNING py.warnings [None req-6daa677d-9c98-4fe0-9ac2-33c7f2232f30 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 26 12:47:32.187598 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: warnings.warn(msg) Jul 26 12:47:32.187598 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: Jul 26 12:47:32.188620 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daa677d-9c98-4fe0-9ac2-33c7f2232f30 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:47:32.201700 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 752/1499] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:47:32.205837 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5bd0d2-a133-40e1-89a1-40709fd90aca admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:47:32.207357 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-da5bd0d2-a133-40e1-89a1-40709fd90aca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:47:32.219212 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5bd0d2-a133-40e1-89a1-40709fd90aca admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:47:32.244546 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:47:32.314384 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160dd9eb-71c4-45c0-a998-1a73a01e3258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:47:32.314922 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160dd9eb-71c4-45c0-a998-1a73a01e3258 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:47:32.315205 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160dd9eb-71c4-45c0-a998-1a73a01e3258 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:47:32.315478 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160dd9eb-71c4-45c0-a998-1a73a01e3258 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:47:32.316283 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160dd9eb-71c4-45c0-a998-1a73a01e3258 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:47:32.317295 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: WARNING py.warnings [None req-160dd9eb-71c4-45c0-a998-1a73a01e3258 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 26 12:47:32.317295 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: warnings.warn(msg) Jul 26 12:47:32.317295 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: Jul 26 12:47:32.318190 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160dd9eb-71c4-45c0-a998-1a73a01e3258 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:47:32.331102 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 748/1500] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:47:32.335311 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daa677d-9c98-4fe0-9ac2-33c7f2232f30 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:47:32.336877 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-6daa677d-9c98-4fe0-9ac2-33c7f2232f30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:47:32.372039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daa677d-9c98-4fe0-9ac2-33c7f2232f30 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:47:32.467125 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28c12f-defb-4f6c-be1d-c43488709ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:47:32.467825 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28c12f-defb-4f6c-be1d-c43488709ff4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:47:32.468109 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28c12f-defb-4f6c-be1d-c43488709ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:47:32.468385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28c12f-defb-4f6c-be1d-c43488709ff4 admin admin] PATH_INFO: `/v3/projects/6b1bf2f0d3304a8691f831eb93261923/users/0f6c71be9a094f74a0b254b62be9f746/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:47:32.490496 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba28c12f-defb-4f6c-be1d-c43488709ff4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b1bf2f0d3304a8691f831eb93261923, user_id=0f6c71be9a094f74a0b254b62be9f746, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:47:32.491524 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: WARNING py.warnings [None req-ba28c12f-defb-4f6c-be1d-c43488709ff4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 12:47:32.491524 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: warnings.warn(msg) Jul 26 12:47:32.491524 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: Jul 26 12:47:32.494154 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba28c12f-defb-4f6c-be1d-c43488709ff4 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:47:32.518170 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 753/1501] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:47:32 2022] PUT /identity/v3/projects/6b1bf2f0d3304a8691f831eb93261923/users/0f6c71be9a094f74a0b254b62be9f746/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:47:32.530275 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f947bee-a84e-4244-9e00-e101e3fda6ff None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:47:32.530564 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f947bee-a84e-4244-9e00-e101e3fda6ff None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:47:32.530833 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f947bee-a84e-4244-9e00-e101e3fda6ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:47:32.587337 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-7f947bee-a84e-4244-9e00-e101e3fda6ff None None] MFA Rules not processed for user `0f6c71be9a094f74a0b254b62be9f746`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:47:32.588479 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-7f947bee-a84e-4244-9e00-e101e3fda6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:47:32.665323 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 749/1502] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:47:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:47:32.678489 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dce6c9-0c3b-49a7-b30c-350721ed1e56 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:47:32.678779 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dce6c9-0c3b-49a7-b30c-350721ed1e56 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:47:32.679058 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dce6c9-0c3b-49a7-b30c-350721ed1e56 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:47:32.681052 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 754/1503] 10.222.0.20 () {54 vars in 964 bytes} [Tue Jul 26 12:47:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:47:32.686141 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b00694-5b7b-4f40-aca3-a4ff16666255 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:47:32.686460 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b00694-5b7b-4f40-aca3-a4ff16666255 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:47:32.686739 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b00694-5b7b-4f40-aca3-a4ff16666255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:47:32.718663 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:47:32.762894 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-a3b00694-5b7b-4f40-aca3-a4ff16666255 None None] MFA Rules not processed for user `9f8ca5e019dd4b9090cee2b2c1d8e15d`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:47:32.764209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-a3b00694-5b7b-4f40-aca3-a4ff16666255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:47:32.839244 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 750/1504] 10.222.0.20 () {58 vars in 1061 bytes} [Tue Jul 26 12:47:32 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:47:32.846951 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dce6c9-0c3b-49a7-b30c-350721ed1e56 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:47:32.848498 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-73dce6c9-0c3b-49a7-b30c-350721ed1e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:47:32.906192 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:47:32.959659 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dce6c9-0c3b-49a7-b30c-350721ed1e56 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:47:33.032716 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7216ec-c8cb-4c5c-9237-4157180fac6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:47:33.033326 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7216ec-c8cb-4c5c-9237-4157180fac6e service cinder] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:47:33.033605 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7216ec-c8cb-4c5c-9237-4157180fac6e service cinder] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:47:33.033884 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7216ec-c8cb-4c5c-9237-4157180fac6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:47:33.035554 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-2a7216ec-c8cb-4c5c-9237-4157180fac6e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:47:33.135375 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7216ec-c8cb-4c5c-9237-4157180fac6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:47:33.136929 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7216ec-c8cb-4c5c-9237-4157180fac6e service cinder] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:47:33.140463 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:47:33.155558 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 755/1505] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:47:32 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:47:33.201609 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85018339-5cd8-4d46-b562-cbd09d84dadc None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:47:33.201893 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85018339-5cd8-4d46-b562-cbd09d84dadc None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:47:33.202194 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85018339-5cd8-4d46-b562-cbd09d84dadc None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:47:33.203469 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 751/1506] 10.222.0.20 () {52 vars in 830 bytes} [Tue Jul 26 12:47:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:47:33.207058 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7216ec-c8cb-4c5c-9237-4157180fac6e service cinder] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:47:33.208300 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-2a7216ec-c8cb-4c5c-9237-4157180fac6e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:47:33.307765 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7216ec-c8cb-4c5c-9237-4157180fac6e service cinder] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:47:33.403780 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b390c-fcaf-4873-a331-046e8db166c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:47:33.404439 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b390c-fcaf-4873-a331-046e8db166c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:47:33.404568 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b390c-fcaf-4873-a331-046e8db166c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:47:33.404793 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b390c-fcaf-4873-a331-046e8db166c6 admin admin] PATH_INFO: `/v3/users/0f6c71be9a094f74a0b254b62be9f746` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:47:33.417249 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22b390c-fcaf-4873-a331-046e8db166c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f6c71be9a094f74a0b254b62be9f746)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:47:33.418284 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: WARNING py.warnings [None req-d22b390c-fcaf-4873-a331-046e8db166c6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 12:47:33.418284 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: warnings.warn(msg) Jul 26 12:47:33.418284 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: Jul 26 12:47:33.419507 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22b390c-fcaf-4873-a331-046e8db166c6 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:47:33.429099 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:47:33.482597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-d22b390c-fcaf-4873-a331-046e8db166c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f6c71be9a094f74a0b254b62be9f746'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:47:33.483199 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-d22b390c-fcaf-4873-a331-046e8db166c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f6c71be9a094f74a0b254b62be9f746'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:47:33.483622 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-d22b390c-fcaf-4873-a331-046e8db166c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f6c71be9a094f74a0b254b62be9f746'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:47:33.502103 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-d22b390c-fcaf-4873-a331-046e8db166c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f6c71be9a094f74a0b254b62be9f746'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:47:33.514430 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-d22b390c-fcaf-4873-a331-046e8db166c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f6c71be9a094f74a0b254b62be9f746'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:47:33.528838 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 756/1507] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:47:33 2022] DELETE /identity/v3/users/0f6c71be9a094f74a0b254b62be9f746 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:47:33.533227 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614fbd5-8e6f-434d-8703-c9ca36daba93 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:47:33.533767 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614fbd5-8e6f-434d-8703-c9ca36daba93 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:47:33.533910 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614fbd5-8e6f-434d-8703-c9ca36daba93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:47:33.591991 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-6614fbd5-8e6f-434d-8703-c9ca36daba93 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:47:33.593205 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-6614fbd5-8e6f-434d-8703-c9ca36daba93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:47:33.669539 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:47:33.683117 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 752/1508] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:47:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:47:33.692624 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b390c-fcaf-4873-a331-046e8db166c6 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:47:33.693858 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-d22b390c-fcaf-4873-a331-046e8db166c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:47:33.791201 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b390c-fcaf-4873-a331-046e8db166c6 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:47:33.845047 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:47:33.865879 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24168c2a-00d0-4ce5-9b21-0e5a2e2b50c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:47:33.866506 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24168c2a-00d0-4ce5-9b21-0e5a2e2b50c1 service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:47:33.866792 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24168c2a-00d0-4ce5-9b21-0e5a2e2b50c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:47:33.867666 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24168c2a-00d0-4ce5-9b21-0e5a2e2b50c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:47:33.869313 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-24168c2a-00d0-4ce5-9b21-0e5a2e2b50c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:47:33.980262 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24168c2a-00d0-4ce5-9b21-0e5a2e2b50c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:47:33.981719 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24168c2a-00d0-4ce5-9b21-0e5a2e2b50c1 service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:47:34.000683 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 757/1509] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:47:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:47:34.360834 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6614fbd5-8e6f-434d-8703-c9ca36daba93 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:47:34.362147 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-6614fbd5-8e6f-434d-8703-c9ca36daba93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:47:34.373706 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6614fbd5-8e6f-434d-8703-c9ca36daba93 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:47:34.474348 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c7639f-5a41-4d9c-a27d-d5141342496d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:47:34.474987 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7639f-5a41-4d9c-a27d-d5141342496d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:47:34.475265 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7639f-5a41-4d9c-a27d-d5141342496d admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:47:34.475535 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7639f-5a41-4d9c-a27d-d5141342496d admin admin] PATH_INFO: `/v3/projects/6b1bf2f0d3304a8691f831eb93261923` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:47:34.486345 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c7639f-5a41-4d9c-a27d-d5141342496d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b1bf2f0d3304a8691f831eb93261923)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:47:34.487457 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: WARNING py.warnings [None req-97c7639f-5a41-4d9c-a27d-d5141342496d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 12:47:34.487457 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: warnings.warn(msg) Jul 26 12:47:34.487457 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: Jul 26 12:47:34.488621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c7639f-5a41-4d9c-a27d-d5141342496d admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:47:34.546740 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-97c7639f-5a41-4d9c-a27d-d5141342496d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b1bf2f0d3304a8691f831eb93261923', 'request_id': 'req-97c7639f-5a41-4d9c-a27d-d5141342496d'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:47:34.560011 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-97c7639f-5a41-4d9c-a27d-d5141342496d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b1bf2f0d3304a8691f831eb93261923', 'request_id': 'req-97c7639f-5a41-4d9c-a27d-d5141342496d'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:47:34.569039 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-97c7639f-5a41-4d9c-a27d-d5141342496d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b1bf2f0d3304a8691f831eb93261923', 'request_id': 'req-97c7639f-5a41-4d9c-a27d-d5141342496d'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:47:34.592722 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-97c7639f-5a41-4d9c-a27d-d5141342496d admin admin] The token cache is being invalidate because project 6b1bf2f0d3304a8691f831eb93261923 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:47:34.593007 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-97c7639f-5a41-4d9c-a27d-d5141342496d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:47:34.593525 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-97c7639f-5a41-4d9c-a27d-d5141342496d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:47:34.594858 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 753/1510] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:47:34 2022] DELETE /identity/v3/projects/6b1bf2f0d3304a8691f831eb93261923 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:47:43.998357 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=62 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:47:44.590739 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:49:12.725530 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af84391f-4505-43c9-801c-48190f29dcfa None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:49:12.726680 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af84391f-4505-43c9-801c-48190f29dcfa None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:49:12.726680 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af84391f-4505-43c9-801c-48190f29dcfa None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:49:12.728141 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 758/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 12:49:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:49:12.733174 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb654b4-adef-4602-b103-85f6919b23df None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:49:12.733735 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb654b4-adef-4602-b103-85f6919b23df None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:49:12.733735 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb654b4-adef-4602-b103-85f6919b23df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:49:12.815316 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-feb654b4-adef-4602-b103-85f6919b23df None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:49:12.816547 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-feb654b4-adef-4602-b103-85f6919b23df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:49:12.927121 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 754/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 12:49:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:49:13.082578 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af84391f-4505-43c9-801c-48190f29dcfa None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:49:13.084759 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-af84391f-4505-43c9-801c-48190f29dcfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:49:13.204678 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af84391f-4505-43c9-801c-48190f29dcfa None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:49:13.211664 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a8a53-c519-469e-aae7-416f35fd6979 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:49:13.212283 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a8a53-c519-469e-aae7-416f35fd6979 service nova] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:49:13.212565 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a8a53-c519-469e-aae7-416f35fd6979 service nova] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:49:13.212840 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a8a53-c519-469e-aae7-416f35fd6979 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:49:13.221932 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165a8a53-c519-469e-aae7-416f35fd6979 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:49:13.223721 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165a8a53-c519-469e-aae7-416f35fd6979 service nova] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:49:13.233213 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 759/1513] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:49:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:49:14.064398 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398cb68-9146-482e-bb9d-b1c8857e153f None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:49:14.064970 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398cb68-9146-482e-bb9d-b1c8857e153f None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:49:14.064970 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398cb68-9146-482e-bb9d-b1c8857e153f None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:49:14.066369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 755/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 12:49:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:49:14.071429 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1c9fb-d3c9-46f7-8c11-0e8955f8a922 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:49:14.071992 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1c9fb-d3c9-46f7-8c11-0e8955f8a922 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:49:14.072490 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1c9fb-d3c9-46f7-8c11-0e8955f8a922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:49:14.092698 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-0eb1c9fb-d3c9-46f7-8c11-0e8955f8a922 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:49:14.093905 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-0eb1c9fb-d3c9-46f7-8c11-0e8955f8a922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:49:14.102397 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 760/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 12:49:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:49:14.256244 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398cb68-9146-482e-bb9d-b1c8857e153f None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:49:14.273460 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398cb68-9146-482e-bb9d-b1c8857e153f None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:49:14.279188 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d44251-0d7d-4ff6-82e5-fd7438f8be41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:49:14.279841 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d44251-0d7d-4ff6-82e5-fd7438f8be41 service nova] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:49:14.280121 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d44251-0d7d-4ff6-82e5-fd7438f8be41 service nova] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:49:14.280393 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d44251-0d7d-4ff6-82e5-fd7438f8be41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:49:14.289343 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d44251-0d7d-4ff6-82e5-fd7438f8be41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:49:14.291046 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d44251-0d7d-4ff6-82e5-fd7438f8be41 service nova] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:49:14.300944 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 756/1516] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:49:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:49:14.815588 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9c027-ec98-4564-a228-ef7a96f915d3 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:49:14.815902 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9c027-ec98-4564-a228-ef7a96f915d3 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:49:14.816184 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9c027-ec98-4564-a228-ef7a96f915d3 None None] PATH_INFO: `/v3` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:49:14.817487 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 761/1517] 10.222.0.46 () {54 vars in 925 bytes} [Tue Jul 26 12:49:14 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:49:14.824453 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ed12d-6584-4e8e-9340-fdfedad46c97 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:49:14.824810 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ed12d-6584-4e8e-9340-fdfedad46c97 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:49:14.825091 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ed12d-6584-4e8e-9340-fdfedad46c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:49:14.890036 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-598ed12d-6584-4e8e-9340-fdfedad46c97 None None] MFA Rules not processed for user `ec49a03aed774c439459883bd5442d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:49:14.895068 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-598ed12d-6584-4e8e-9340-fdfedad46c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:49:14.914073 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:49:14.972644 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 757/1518] 10.222.0.46 () {58 vars in 1015 bytes} [Tue Jul 26 12:49:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:49:18.764160 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83403e-dc22-4cca-8618-431682796c1c None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:49:18.764689 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83403e-dc22-4cca-8618-431682796c1c None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:49:18.764689 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83403e-dc22-4cca-8618-431682796c1c None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:49:18.765992 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 762/1519] 10.222.0.20 () {54 vars in 961 bytes} [Tue Jul 26 12:49:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:49:18.771060 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774a8b3-3ff7-45f2-b549-f68a8ec2db0a None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:49:18.771497 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774a8b3-3ff7-45f2-b549-f68a8ec2db0a None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:49:18.771608 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774a8b3-3ff7-45f2-b549-f68a8ec2db0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:49:18.791709 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-3774a8b3-3ff7-45f2-b549-f68a8ec2db0a None None] MFA Rules not processed for user `ec49a03aed774c439459883bd5442d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:49:18.792916 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-3774a8b3-3ff7-45f2-b549-f68a8ec2db0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:49:18.801495 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 758/1520] 10.222.0.20 () {58 vars in 1058 bytes} [Tue Jul 26 12:49:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:49:18.808852 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c83403e-dc22-4cca-8618-431682796c1c None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:49:18.822423 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c83403e-dc22-4cca-8618-431682796c1c None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:49:18.827223 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573482f2-10df-4c24-9714-6b29ff04a16b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec49a03aed774c439459883bd5442d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:49:18.827838 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573482f2-10df-4c24-9714-6b29ff04a16b service placement] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:49:18.828117 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573482f2-10df-4c24-9714-6b29ff04a16b service placement] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:49:18.828385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573482f2-10df-4c24-9714-6b29ff04a16b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:49:18.836990 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573482f2-10df-4c24-9714-6b29ff04a16b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:49:18.838557 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573482f2-10df-4c24-9714-6b29ff04a16b service placement] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:49:18.847994 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 763/1521] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:49:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:49:25.537879 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe692a-aec7-4430-82c1-5e44240307a0 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:49:25.538480 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe692a-aec7-4430-82c1-5e44240307a0 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:49:25.538480 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe692a-aec7-4430-82c1-5e44240307a0 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:49:25.539902 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 759/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 12:49:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:49:25.544836 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be1e41-c0ee-424b-be08-a26abad85f67 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:49:25.545316 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be1e41-c0ee-424b-be08-a26abad85f67 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:49:25.545420 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be1e41-c0ee-424b-be08-a26abad85f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:49:25.564318 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-81be1e41-c0ee-424b-be08-a26abad85f67 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:49:25.565497 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-81be1e41-c0ee-424b-be08-a26abad85f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:49:25.573775 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 764/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 12:49:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:49:25.728561 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe692a-aec7-4430-82c1-5e44240307a0 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:49:25.742960 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe692a-aec7-4430-82c1-5e44240307a0 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:49:25.748650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f0c09c-710f-4e5d-bb71-40a7c624ffff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:49:25.749235 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0c09c-710f-4e5d-bb71-40a7c624ffff service nova] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:49:25.749514 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0c09c-710f-4e5d-bb71-40a7c624ffff service nova] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:49:25.749786 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0c09c-710f-4e5d-bb71-40a7c624ffff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:49:25.758248 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f0c09c-710f-4e5d-bb71-40a7c624ffff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:49:25.759861 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f0c09c-710f-4e5d-bb71-40a7c624ffff service nova] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:49:25.769018 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 760/1524] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:49:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:49:26.594241 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b606ecb1-868b-4b74-808b-1e38a7f1c06d None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:49:26.594855 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b606ecb1-868b-4b74-808b-1e38a7f1c06d None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:49:26.594855 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b606ecb1-868b-4b74-808b-1e38a7f1c06d None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:49:26.596195 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 765/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 12:49:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:49:26.601527 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633aac92-3d2b-4340-9ca2-787b4ab0fc7e None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:49:26.601989 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633aac92-3d2b-4340-9ca2-787b4ab0fc7e None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:49:26.602101 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633aac92-3d2b-4340-9ca2-787b4ab0fc7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:49:26.622273 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-633aac92-3d2b-4340-9ca2-787b4ab0fc7e None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:49:26.623456 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-633aac92-3d2b-4340-9ca2-787b4ab0fc7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:49:26.631314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 761/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 12:49:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:49:26.781631 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b606ecb1-868b-4b74-808b-1e38a7f1c06d None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:49:26.796640 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b606ecb1-868b-4b74-808b-1e38a7f1c06d None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:49:26.801973 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d94777c-f60c-475b-b68d-dee2f6b34d92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:49:26.802564 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d94777c-f60c-475b-b68d-dee2f6b34d92 service nova] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:49:26.802842 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d94777c-f60c-475b-b68d-dee2f6b34d92 service nova] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:49:26.803112 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d94777c-f60c-475b-b68d-dee2f6b34d92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:49:26.811093 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d94777c-f60c-475b-b68d-dee2f6b34d92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:49:26.812725 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d94777c-f60c-475b-b68d-dee2f6b34d92 service nova] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:49:26.822009 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 766/1527] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:49:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:49:30.713043 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7b8d3-f72c-4e78-acd2-50a9490c515c None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:49:30.713644 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7b8d3-f72c-4e78-acd2-50a9490c515c None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:49:30.713644 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7b8d3-f72c-4e78-acd2-50a9490c515c None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:49:30.715145 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 762/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 12:49:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:49:30.720074 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f9f8d1-0327-42f5-84b6-128573df1253 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:49:30.720489 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f9f8d1-0327-42f5-84b6-128573df1253 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:49:30.720622 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f9f8d1-0327-42f5-84b6-128573df1253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:49:30.739409 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-c0f9f8d1-0327-42f5-84b6-128573df1253 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:49:30.740623 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-c0f9f8d1-0327-42f5-84b6-128573df1253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:49:30.749086 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 767/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 12:49:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:49:30.822948 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f7b8d3-f72c-4e78-acd2-50a9490c515c None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:49:30.824699 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-96f7b8d3-f72c-4e78-acd2-50a9490c515c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:49:30.907577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f7b8d3-f72c-4e78-acd2-50a9490c515c None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:49:30.911247 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d7193-0680-48d7-aa74-c3f5183d3b7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:49:30.911926 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d7193-0680-48d7-aa74-c3f5183d3b7e service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:49:30.912211 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d7193-0680-48d7-aa74-c3f5183d3b7e service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:49:30.912485 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d7193-0680-48d7-aa74-c3f5183d3b7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:49:30.922179 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d7193-0680-48d7-aa74-c3f5183d3b7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:49:30.923612 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d7193-0680-48d7-aa74-c3f5183d3b7e service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:49:30.933549 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 763/1530] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:49:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:49:32.971086 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f6faa-edf7-481f-8043-58df3efad21d None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:49:32.971687 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f6faa-edf7-481f-8043-58df3efad21d None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:49:32.971884 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f6faa-edf7-481f-8043-58df3efad21d None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:49:32.973584 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 768/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 12:49:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:49:32.979658 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5caf9a4-02f1-4a96-ad6a-485fe41d4e2c None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:49:32.980153 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5caf9a4-02f1-4a96-ad6a-485fe41d4e2c None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:49:32.980257 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5caf9a4-02f1-4a96-ad6a-485fe41d4e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:49:33.000330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-e5caf9a4-02f1-4a96-ad6a-485fe41d4e2c None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:49:33.001542 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-e5caf9a4-02f1-4a96-ad6a-485fe41d4e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:49:33.010018 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 764/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 12:49:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:49:33.046859 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f6faa-edf7-481f-8043-58df3efad21d None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:49:33.058179 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f6faa-edf7-481f-8043-58df3efad21d None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:49:33.061682 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79e9e4c-5115-46e4-ab41-00d94d054ed0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:49:33.062273 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e9e4c-5115-46e4-ab41-00d94d054ed0 service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:49:33.062551 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e9e4c-5115-46e4-ab41-00d94d054ed0 service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:49:33.062820 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e9e4c-5115-46e4-ab41-00d94d054ed0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:49:33.071622 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79e9e4c-5115-46e4-ab41-00d94d054ed0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:49:33.073081 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79e9e4c-5115-46e4-ab41-00d94d054ed0 service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:49:33.082555 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 769/1533] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:49:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 12:49:36.563803 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a876042b-7be2-4b8b-b40b-9b10251d4826 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:49:36.564380 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a876042b-7be2-4b8b-b40b-9b10251d4826 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:49:36.564380 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a876042b-7be2-4b8b-b40b-9b10251d4826 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:49:36.565697 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 765/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 12:49:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:49:36.570841 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4149cce-2cbd-4d02-bc12-37a8db692f77 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:49:36.571263 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4149cce-2cbd-4d02-bc12-37a8db692f77 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:49:36.571384 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4149cce-2cbd-4d02-bc12-37a8db692f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:49:36.579214 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=81 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:49:36.592309 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-a4149cce-2cbd-4d02-bc12-37a8db692f77 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:49:36.593516 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-a4149cce-2cbd-4d02-bc12-37a8db692f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:49:36.601947 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 770/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 12:49:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:49:36.904882 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a876042b-7be2-4b8b-b40b-9b10251d4826 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:49:36.906561 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-a876042b-7be2-4b8b-b40b-9b10251d4826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:49:36.970361 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:49:36.984807 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a876042b-7be2-4b8b-b40b-9b10251d4826 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:49:36.988428 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494896aa-83d6-49c3-a030-788dc400131e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:49:36.989036 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494896aa-83d6-49c3-a030-788dc400131e service cinder] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:49:36.989312 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494896aa-83d6-49c3-a030-788dc400131e service cinder] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:49:36.989582 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494896aa-83d6-49c3-a030-788dc400131e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:49:36.998646 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494896aa-83d6-49c3-a030-788dc400131e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:49:37.000108 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494896aa-83d6-49c3-a030-788dc400131e service cinder] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:49:37.009752 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 766/1536] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:49:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:49:39.040385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8657ea41-fcec-476d-b2c3-3f2cbbc7b941 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:49:39.040957 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8657ea41-fcec-476d-b2c3-3f2cbbc7b941 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:49:39.040957 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8657ea41-fcec-476d-b2c3-3f2cbbc7b941 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:49:39.042303 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 771/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 12:49:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:49:39.047464 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f863f-4c5f-4d94-bcd8-b420d8a5b0cd None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:49:39.047917 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f863f-4c5f-4d94-bcd8-b420d8a5b0cd None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:49:39.048040 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f863f-4c5f-4d94-bcd8-b420d8a5b0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:49:39.071164 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-022f863f-4c5f-4d94-bcd8-b420d8a5b0cd None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:49:39.072375 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-022f863f-4c5f-4d94-bcd8-b420d8a5b0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:49:39.080159 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 767/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 12:49:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:49:39.381403 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8657ea41-fcec-476d-b2c3-3f2cbbc7b941 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:49:39.383071 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-8657ea41-fcec-476d-b2c3-3f2cbbc7b941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:49:39.395660 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8657ea41-fcec-476d-b2c3-3f2cbbc7b941 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:49:39.399625 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82813633-6918-43e7-871a-2dc13de1791d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:49:39.400241 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82813633-6918-43e7-871a-2dc13de1791d service cinder] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:49:39.400517 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82813633-6918-43e7-871a-2dc13de1791d service cinder] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:49:39.400787 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82813633-6918-43e7-871a-2dc13de1791d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:49:39.409459 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82813633-6918-43e7-871a-2dc13de1791d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:49:39.410868 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82813633-6918-43e7-871a-2dc13de1791d service cinder] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:49:39.420605 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 772/1539] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:49:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:49:49.066160 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=11 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:49:49.416668 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=8 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:08.363967 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc34407-71a8-4a0c-861d-8658f5fffbce None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:08.364613 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc34407-71a8-4a0c-861d-8658f5fffbce None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:08.364613 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc34407-71a8-4a0c-861d-8658f5fffbce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:08.389450 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-4cc34407-71a8-4a0c-861d-8658f5fffbce None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:08.390674 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-4cc34407-71a8-4a0c-861d-8658f5fffbce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:08.406909 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 768/1540] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:08.416935 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe44908-fd23-40f6-8393-514ac2123e3c None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:08.417490 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe44908-fd23-40f6-8393-514ac2123e3c None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:08.417490 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe44908-fd23-40f6-8393-514ac2123e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:08.443910 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-5fe44908-fd23-40f6-8393-514ac2123e3c None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:08.445081 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-5fe44908-fd23-40f6-8393-514ac2123e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:08.458297 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 773/1541] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:08.463775 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc34407-71a8-4a0c-861d-8658f5fffbce None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:08.474567 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc34407-71a8-4a0c-861d-8658f5fffbce None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:08.483092 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8799b17a-cbe4-468c-abfe-d71ff7bf6b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:08.483761 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799b17a-cbe4-468c-abfe-d71ff7bf6b90 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:08.484044 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799b17a-cbe4-468c-abfe-d71ff7bf6b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:08.484318 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799b17a-cbe4-468c-abfe-d71ff7bf6b90 admin admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:08.485025 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8799b17a-cbe4-468c-abfe-d71ff7bf6b90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:51:08.485330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8799b17a-cbe4-468c-abfe-d71ff7bf6b90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:08.487093 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8799b17a-cbe4-468c-abfe-d71ff7bf6b90 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:08.503143 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50569b-515f-4c53-9714-0af14b763e4a None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:08.503467 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50569b-515f-4c53-9714-0af14b763e4a None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:08.503725 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50569b-515f-4c53-9714-0af14b763e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:08.505116 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 769/1542] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:51:08.513550 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8799b17a-cbe4-468c-abfe-d71ff7bf6b90 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:08.526235 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8799b17a-cbe4-468c-abfe-d71ff7bf6b90 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:08.535974 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-9e50569b-515f-4c53-9714-0af14b763e4a None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:08.537541 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-9e50569b-515f-4c53-9714-0af14b763e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:08.540450 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6b9782-b62d-4699-8514-8a4100879e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:08.541266 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b9782-b62d-4699-8514-8a4100879e04 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:08.541550 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b9782-b62d-4699-8514-8a4100879e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:08.541818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b9782-b62d-4699-8514-8a4100879e04 admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:08.542487 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6b9782-b62d-4699-8514-8a4100879e04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:08.543512 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: WARNING py.warnings [None req-5b6b9782-b62d-4699-8514-8a4100879e04 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 12:51:08.543512 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: warnings.warn(msg) Jul 26 12:51:08.543512 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: Jul 26 12:51:08.545512 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6b9782-b62d-4699-8514-8a4100879e04 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:08.554708 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 774/1543] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:51:08.563325 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8441c9b4-695e-41a0-9f23-fccd69e66c51 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:08.563949 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8441c9b4-695e-41a0-9f23-fccd69e66c51 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:08.564870 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8441c9b4-695e-41a0-9f23-fccd69e66c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:08.609603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-8441c9b4-695e-41a0-9f23-fccd69e66c51 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:08.612978 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-8441c9b4-695e-41a0-9f23-fccd69e66c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:08.616823 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 770/1544] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:51:08 2022] POST /identity/v3/projects => generated 391 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:51:08.623996 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019b7c1-2161-4cfd-9b5a-56c39911fed3 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:08.624805 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019b7c1-2161-4cfd-9b5a-56c39911fed3 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:08.625090 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019b7c1-2161-4cfd-9b5a-56c39911fed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:08.665771 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-7019b7c1-2161-4cfd-9b5a-56c39911fed3 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:08.667383 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-7019b7c1-2161-4cfd-9b5a-56c39911fed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:08.720665 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 775/1545] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:08.721724 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8441c9b4-695e-41a0-9f23-fccd69e66c51 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:08.736550 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8441c9b4-695e-41a0-9f23-fccd69e66c51 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:08.739239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 771/1546] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:08.744828 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f4383-f121-40f7-85ff-e20e684d4ffd None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:08.746407 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f4383-f121-40f7-85ff-e20e684d4ffd None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:08.747857 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f4383-f121-40f7-85ff-e20e684d4ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:08.748697 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7070b52b-a49b-4c5c-887d-07847ff53b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:08.749323 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070b52b-a49b-4c5c-887d-07847ff53b14 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:08.749603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070b52b-a49b-4c5c-887d-07847ff53b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:08.749878 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070b52b-a49b-4c5c-887d-07847ff53b14 admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:08.750556 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7070b52b-a49b-4c5c-887d-07847ff53b14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:08.754665 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: WARNING py.warnings [None req-7070b52b-a49b-4c5c-887d-07847ff53b14 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 12:51:08.754665 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: warnings.warn(msg) Jul 26 12:51:08.754665 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: Jul 26 12:51:08.755924 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7070b52b-a49b-4c5c-887d-07847ff53b14 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:08.774408 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-691f4383-f121-40f7-85ff-e20e684d4ffd None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:08.775785 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-691f4383-f121-40f7-85ff-e20e684d4ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:08.787154 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 772/1547] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:08.788278 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411756a7-8d7a-48e5-a2ce-8f999fdcbd09 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:08.789110 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411756a7-8d7a-48e5-a2ce-8f999fdcbd09 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:08.789397 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411756a7-8d7a-48e5-a2ce-8f999fdcbd09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:08.791910 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 776/1548] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:51:08 2022] POST /identity/v3/users => generated 296 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:51:08.793290 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87157de0-5cc0-4445-a523-94e8ca3ecd01 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:08.793615 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87157de0-5cc0-4445-a523-94e8ca3ecd01 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:08.793855 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87157de0-5cc0-4445-a523-94e8ca3ecd01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:08.809966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-411756a7-8d7a-48e5-a2ce-8f999fdcbd09 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:08.811185 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-411756a7-8d7a-48e5-a2ce-8f999fdcbd09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:08.813107 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-87157de0-5cc0-4445-a523-94e8ca3ecd01 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:08.814270 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-87157de0-5cc0-4445-a523-94e8ca3ecd01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:08.819535 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 773/1549] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:08.820416 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411756a7-8d7a-48e5-a2ce-8f999fdcbd09 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:08.827626 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 777/1550] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:08.827626 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357e2ad-9501-4df1-b328-fa5f5b316b1c None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:08.827626 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357e2ad-9501-4df1-b328-fa5f5b316b1c None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:08.827626 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357e2ad-9501-4df1-b328-fa5f5b316b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:08.830423 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411756a7-8d7a-48e5-a2ce-8f999fdcbd09 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:08.835714 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fcc290-c216-474d-a6ba-1386ddc3cf37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:08.835930 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fcc290-c216-474d-a6ba-1386ddc3cf37 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:08.836212 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fcc290-c216-474d-a6ba-1386ddc3cf37 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:08.836490 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fcc290-c216-474d-a6ba-1386ddc3cf37 admin admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:08.837195 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fcc290-c216-474d-a6ba-1386ddc3cf37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:51:08.837523 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fcc290-c216-474d-a6ba-1386ddc3cf37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:08.839137 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fcc290-c216-474d-a6ba-1386ddc3cf37 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:08.845589 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-f357e2ad-9501-4df1-b328-fa5f5b316b1c None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:08.846782 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-f357e2ad-9501-4df1-b328-fa5f5b316b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:08.851585 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 774/1551] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:51:08.853129 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7915aa2-1f90-4b03-b485-3d040bc11845 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:08.853416 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7915aa2-1f90-4b03-b485-3d040bc11845 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:08.853693 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7915aa2-1f90-4b03-b485-3d040bc11845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:08.855608 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 778/1552] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:08.856894 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196388d-1833-4c4a-b67b-9f190f26b360 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:08.857157 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196388d-1833-4c4a-b67b-9f190f26b360 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:08.857436 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196388d-1833-4c4a-b67b-9f190f26b360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:08.876699 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-b7915aa2-1f90-4b03-b485-3d040bc11845 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:08.877242 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-f196388d-1833-4c4a-b67b-9f190f26b360 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:08.877906 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-b7915aa2-1f90-4b03-b485-3d040bc11845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:08.878386 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-f196388d-1833-4c4a-b67b-9f190f26b360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:08.893257 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 779/1554] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:08.893484 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 775/1554] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:08.893609 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196388d-1833-4c4a-b67b-9f190f26b360 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:08.893696 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1dacd4-0c09-4743-bc3d-7ae81abb64c6 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:08.893696 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1dacd4-0c09-4743-bc3d-7ae81abb64c6 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:08.893696 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1dacd4-0c09-4743-bc3d-7ae81abb64c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:08.902350 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196388d-1833-4c4a-b67b-9f190f26b360 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:08.906953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c93b86-f793-4f0f-ae17-38dc9dfc7039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:08.907586 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c93b86-f793-4f0f-ae17-38dc9dfc7039 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:08.907836 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-cf1dacd4-0c09-4743-bc3d-7ae81abb64c6 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:08.907942 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c93b86-f793-4f0f-ae17-38dc9dfc7039 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:08.908154 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c93b86-f793-4f0f-ae17-38dc9dfc7039 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:08.908845 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c93b86-f793-4f0f-ae17-38dc9dfc7039 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:08.908959 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-cf1dacd4-0c09-4743-bc3d-7ae81abb64c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:08.910469 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c93b86-f793-4f0f-ae17-38dc9dfc7039 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:08.917374 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 776/1555] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:08.918588 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaaa79f-927a-4d3e-b401-d225774a8ce5 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:08.918873 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaaa79f-927a-4d3e-b401-d225774a8ce5 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:08.919147 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaaa79f-927a-4d3e-b401-d225774a8ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:08.924354 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 780/1556] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:51:08.924354 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d60fa-3a8f-495d-b2db-ff35b514010b None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:08.924354 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d60fa-3a8f-495d-b2db-ff35b514010b None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:08.924354 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d60fa-3a8f-495d-b2db-ff35b514010b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:08.938771 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-eaaaa79f-927a-4d3e-b401-d225774a8ce5 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:08.939994 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-eaaaa79f-927a-4d3e-b401-d225774a8ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:08.942680 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-d94d60fa-3a8f-495d-b2db-ff35b514010b None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:08.943887 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-d94d60fa-3a8f-495d-b2db-ff35b514010b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:08.947873 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 777/1557] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:08.948632 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaaa79f-927a-4d3e-b401-d225774a8ce5 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:08.951795 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 781/1558] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:08.952581 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d60fa-3a8f-495d-b2db-ff35b514010b None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:08.960996 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaaa79f-927a-4d3e-b401-d225774a8ce5 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:08.962199 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d60fa-3a8f-495d-b2db-ff35b514010b None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:08.966030 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee52b83c-2dd0-4aac-a8a2-9b105d87ad9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:08.966663 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52b83c-2dd0-4aac-a8a2-9b105d87ad9b admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:08.966835 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab64c4-52c6-4e56-a93b-7c7f81ddcad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:08.966965 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52b83c-2dd0-4aac-a8a2-9b105d87ad9b admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:08.967193 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52b83c-2dd0-4aac-a8a2-9b105d87ad9b admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:08.967427 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab64c4-52c6-4e56-a93b-7c7f81ddcad6 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:08.967737 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab64c4-52c6-4e56-a93b-7c7f81ddcad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:08.967884 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee52b83c-2dd0-4aac-a8a2-9b105d87ad9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:08.968015 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab64c4-52c6-4e56-a93b-7c7f81ddcad6 admin admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:08.968703 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ab64c4-52c6-4e56-a93b-7c7f81ddcad6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:51:08.969643 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ab64c4-52c6-4e56-a93b-7c7f81ddcad6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:08.969775 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee52b83c-2dd0-4aac-a8a2-9b105d87ad9b admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:08.970015 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: WARNING py.warnings [None req-38ab64c4-52c6-4e56-a93b-7c7f81ddcad6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 26 12:51:08.970015 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: warnings.warn(msg) Jul 26 12:51:08.970015 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: Jul 26 12:51:08.970954 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ab64c4-52c6-4e56-a93b-7c7f81ddcad6 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:08.983724 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 782/1559] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:51:08.984980 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e216b7b-5375-4a64-b77e-b7a9956584c0 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:08.985305 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e216b7b-5375-4a64-b77e-b7a9956584c0 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:08.985588 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e216b7b-5375-4a64-b77e-b7a9956584c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:09.009052 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 778/1560] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:51:08 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:51:09.009052 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c83c460-881d-46b3-a1ba-aa40b6a5a7cd None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:09.009052 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c83c460-881d-46b3-a1ba-aa40b6a5a7cd None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:09.009052 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c83c460-881d-46b3-a1ba-aa40b6a5a7cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:09.009502 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-2e216b7b-5375-4a64-b77e-b7a9956584c0 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:09.010011 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-2e216b7b-5375-4a64-b77e-b7a9956584c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:09.027104 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-7c83c460-881d-46b3-a1ba-aa40b6a5a7cd None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:09.028461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-7c83c460-881d-46b3-a1ba-aa40b6a5a7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:09.068880 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:09.069471 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 783/1561] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:09.070354 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e216b7b-5375-4a64-b77e-b7a9956584c0 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:09.082557 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e216b7b-5375-4a64-b77e-b7a9956584c0 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:09.088070 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1116dffe-5f29-40e0-bb95-09a0190f4e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:09.088687 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116dffe-5f29-40e0-bb95-09a0190f4e02 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:09.088970 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116dffe-5f29-40e0-bb95-09a0190f4e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:09.089248 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116dffe-5f29-40e0-bb95-09a0190f4e02 admin admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:09.089945 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1116dffe-5f29-40e0-bb95-09a0190f4e02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:51:09.090253 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1116dffe-5f29-40e0-bb95-09a0190f4e02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:09.094313 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 779/1562] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:09.094313 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c83c460-881d-46b3-a1ba-aa40b6a5a7cd None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:09.094537 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1116dffe-5f29-40e0-bb95-09a0190f4e02 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:09.100416 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c83c460-881d-46b3-a1ba-aa40b6a5a7cd None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:09.105045 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a662cc1-9afa-4e10-ba99-24c842526b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:09.105275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 784/1563] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:51:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:51:09.105638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a662cc1-9afa-4e10-ba99-24c842526b54 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:09.105917 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a662cc1-9afa-4e10-ba99-24c842526b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:09.106083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1116dffe-5f29-40e0-bb95-09a0190f4e02 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:09.106203 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a662cc1-9afa-4e10-ba99-24c842526b54 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:09.106861 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a662cc1-9afa-4e10-ba99-24c842526b54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:09.108496 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a662cc1-9afa-4e10-ba99-24c842526b54 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:09.122671 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 780/1564] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:51:09.122671 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a662cc1-9afa-4e10-ba99-24c842526b54 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:09.122933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1116dffe-5f29-40e0-bb95-09a0190f4e02 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:09.127386 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f5de5b-ff1c-4ddc-b38f-8ceb857c2b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:09.128034 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5de5b-ff1c-4ddc-b38f-8ceb857c2b0b admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:09.128314 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5de5b-ff1c-4ddc-b38f-8ceb857c2b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:09.128589 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5de5b-ff1c-4ddc-b38f-8ceb857c2b0b admin admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:09.129285 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f5de5b-ff1c-4ddc-b38f-8ceb857c2b0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:51:09.129783 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f5de5b-ff1c-4ddc-b38f-8ceb857c2b0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:09.131436 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f5de5b-ff1c-4ddc-b38f-8ceb857c2b0b admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:09.132432 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a662cc1-9afa-4e10-ba99-24c842526b54 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:09.133259 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:09.137925 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dce075-943c-466c-a17f-58203eec6d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:09.138559 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dce075-943c-466c-a17f-58203eec6d53 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:09.138846 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dce075-943c-466c-a17f-58203eec6d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:09.139121 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dce075-943c-466c-a17f-58203eec6d53 admin admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:09.139856 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dce075-943c-466c-a17f-58203eec6d53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:51:09.140164 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dce075-943c-466c-a17f-58203eec6d53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:09.141791 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dce075-943c-466c-a17f-58203eec6d53 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:09.145486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 785/1565] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:51:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:51:09.146643 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f5de5b-ff1c-4ddc-b38f-8ceb857c2b0b admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:09.156386 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 781/1566] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:51:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:51:09.157438 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dce075-943c-466c-a17f-58203eec6d53 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:09.158217 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f5de5b-ff1c-4ddc-b38f-8ceb857c2b0b admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:09.167305 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deee536e-f26f-4e4e-9ab5-5d0ab8d14114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:09.167951 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee536e-f26f-4e4e-9ab5-5d0ab8d14114 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:09.168205 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dce075-943c-466c-a17f-58203eec6d53 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:09.168336 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee536e-f26f-4e4e-9ab5-5d0ab8d14114 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:09.168488 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee536e-f26f-4e4e-9ab5-5d0ab8d14114 admin admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:09.169195 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deee536e-f26f-4e4e-9ab5-5d0ab8d14114 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:09.171096 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deee536e-f26f-4e4e-9ab5-5d0ab8d14114 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:09.172962 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16617c3-96fa-49a7-bc09-7bc45393c08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:09.173564 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16617c3-96fa-49a7-bc09-7bc45393c08a admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:09.173840 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16617c3-96fa-49a7-bc09-7bc45393c08a admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:09.175410 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16617c3-96fa-49a7-bc09-7bc45393c08a admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:09.175410 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16617c3-96fa-49a7-bc09-7bc45393c08a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:09.176686 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16617c3-96fa-49a7-bc09-7bc45393c08a admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:09.199153 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 782/1567] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:51:09 2022] POST /identity/v3/users => generated 286 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:51:09.200410 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16617c3-96fa-49a7-bc09-7bc45393c08a admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:09.208111 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 786/1568] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:51:09 2022] POST /identity/v3/projects => generated 385 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:51:09.209070 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deee536e-f26f-4e4e-9ab5-5d0ab8d14114 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:09.210973 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16617c3-96fa-49a7-bc09-7bc45393c08a admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:09.268617 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456149f6-edfb-4f96-92a8-e6565b324165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:09.269161 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456149f6-edfb-4f96-92a8-e6565b324165 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:09.270071 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456149f6-edfb-4f96-92a8-e6565b324165 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:09.270352 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456149f6-edfb-4f96-92a8-e6565b324165 admin admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:09.271056 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456149f6-edfb-4f96-92a8-e6565b324165 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:51:09.271363 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456149f6-edfb-4f96-92a8-e6565b324165 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:09.273596 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456149f6-edfb-4f96-92a8-e6565b324165 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:09.276249 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deee536e-f26f-4e4e-9ab5-5d0ab8d14114 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:09.281100 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8818630e-7e1f-4f2d-8559-d2927550655a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:09.281713 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818630e-7e1f-4f2d-8559-d2927550655a admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:09.281993 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818630e-7e1f-4f2d-8559-d2927550655a admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:09.282296 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818630e-7e1f-4f2d-8559-d2927550655a admin admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:09.282982 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8818630e-7e1f-4f2d-8559-d2927550655a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:51:09.283285 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8818630e-7e1f-4f2d-8559-d2927550655a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:09.284918 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8818630e-7e1f-4f2d-8559-d2927550655a admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:09.285526 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 783/1569] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:51:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:51:09.286322 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456149f6-edfb-4f96-92a8-e6565b324165 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:09.296174 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456149f6-edfb-4f96-92a8-e6565b324165 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:09.307937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 787/1570] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:51:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:51:09.308138 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1b205f-b4ca-4052-a9d6-42f5b3b9adc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:09.308138 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1b205f-b4ca-4052-a9d6-42f5b3b9adc4 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:09.308354 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8818630e-7e1f-4f2d-8559-d2927550655a admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:09.308451 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1b205f-b4ca-4052-a9d6-42f5b3b9adc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:09.308451 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1b205f-b4ca-4052-a9d6-42f5b3b9adc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:09.308451 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1b205f-b4ca-4052-a9d6-42f5b3b9adc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:09.308451 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1b205f-b4ca-4052-a9d6-42f5b3b9adc4 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:09.314263 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8818630e-7e1f-4f2d-8559-d2927550655a admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:09.318917 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fe343f-18fc-4a2d-94bb-1f3dfcfc7d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:09.319571 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe343f-18fc-4a2d-94bb-1f3dfcfc7d50 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:09.319940 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe343f-18fc-4a2d-94bb-1f3dfcfc7d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:09.320211 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe343f-18fc-4a2d-94bb-1f3dfcfc7d50 admin admin] PATH_INFO: `/v3/projects/d92bfe73c0a54823a88a89f9ae627134/users/2b5894762d4b4cc99521bc224d2bddd7/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:09.332991 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 784/1571] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:51:09 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:51:09.333805 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1b205f-b4ca-4052-a9d6-42f5b3b9adc4 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:09.338280 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fe343f-18fc-4a2d-94bb-1f3dfcfc7d50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d92bfe73c0a54823a88a89f9ae627134, user_id=2b5894762d4b4cc99521bc224d2bddd7, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:09.341056 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fe343f-18fc-4a2d-94bb-1f3dfcfc7d50 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:09.354929 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 788/1572] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:51:09 2022] PUT /identity/v3/projects/d92bfe73c0a54823a88a89f9ae627134/users/2b5894762d4b4cc99521bc224d2bddd7/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:09.355736 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fe343f-18fc-4a2d-94bb-1f3dfcfc7d50 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:09.400755 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1b205f-b4ca-4052-a9d6-42f5b3b9adc4 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:09.416887 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:09.418008 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fe343f-18fc-4a2d-94bb-1f3dfcfc7d50 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:09.423276 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7baf3-a667-406e-9f4d-d2591912be25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:09.423915 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7baf3-a667-406e-9f4d-d2591912be25 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:09.424323 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7baf3-a667-406e-9f4d-d2591912be25 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:09.424605 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7baf3-a667-406e-9f4d-d2591912be25 admin admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:09.425267 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd7baf3-a667-406e-9f4d-d2591912be25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:09.427451 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd7baf3-a667-406e-9f4d-d2591912be25 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:09.456728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 789/1573] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:51:09 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:51:09.456728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7baf3-a667-406e-9f4d-d2591912be25 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:09.462134 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ed55f1-3d55-4f13-aeb0-a0490b26611b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:09.462763 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed55f1-3d55-4f13-aeb0-a0490b26611b admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:09.463045 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed55f1-3d55-4f13-aeb0-a0490b26611b admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:09.463322 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed55f1-3d55-4f13-aeb0-a0490b26611b admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:09.464051 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ed55f1-3d55-4f13-aeb0-a0490b26611b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:09.465977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ed55f1-3d55-4f13-aeb0-a0490b26611b admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:09.488128 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:09.495162 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 785/1574] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:51:09 2022] POST /identity/v3/projects => generated 377 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:51:09.496011 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ed55f1-3d55-4f13-aeb0-a0490b26611b admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:09.524771 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7baf3-a667-406e-9f4d-d2591912be25 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:09.560471 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ed55f1-3d55-4f13-aeb0-a0490b26611b admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:09.565314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266d987-8e89-4bdf-b62b-5017efadef08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:09.565937 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266d987-8e89-4bdf-b62b-5017efadef08 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:09.566215 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266d987-8e89-4bdf-b62b-5017efadef08 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:09.566494 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266d987-8e89-4bdf-b62b-5017efadef08 admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:09.567155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d266d987-8e89-4bdf-b62b-5017efadef08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:09.569120 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d266d987-8e89-4bdf-b62b-5017efadef08 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:09.580476 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982cc11a-7fe3-4c18-abb9-112f9973d5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:09.581094 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982cc11a-7fe3-4c18-abb9-112f9973d5c1 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:09.581372 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982cc11a-7fe3-4c18-abb9-112f9973d5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:09.581690 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982cc11a-7fe3-4c18-abb9-112f9973d5c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:09.582483 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982cc11a-7fe3-4c18-abb9-112f9973d5c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:09.584357 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982cc11a-7fe3-4c18-abb9-112f9973d5c1 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:09.593037 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 786/1575] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:51:09 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:51:09.594098 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266d987-8e89-4bdf-b62b-5017efadef08 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:09.595526 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 790/1576] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:51:09.596417 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982cc11a-7fe3-4c18-abb9-112f9973d5c1 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:09.606269 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266d987-8e89-4bdf-b62b-5017efadef08 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:09.606697 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982cc11a-7fe3-4c18-abb9-112f9973d5c1 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:09.611514 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3225bd-579d-45c9-91da-a1bccfcf7c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:09.611694 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24030c5f-dd1e-4639-8fb5-7fcc2df86f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:09.612141 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3225bd-579d-45c9-91da-a1bccfcf7c83 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:09.612267 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24030c5f-dd1e-4639-8fb5-7fcc2df86f64 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:09.612420 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3225bd-579d-45c9-91da-a1bccfcf7c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:09.612516 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24030c5f-dd1e-4639-8fb5-7fcc2df86f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:09.612690 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3225bd-579d-45c9-91da-a1bccfcf7c83 admin admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:09.612794 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24030c5f-dd1e-4639-8fb5-7fcc2df86f64 admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:09.613347 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3225bd-579d-45c9-91da-a1bccfcf7c83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:09.613443 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24030c5f-dd1e-4639-8fb5-7fcc2df86f64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:09.615241 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3225bd-579d-45c9-91da-a1bccfcf7c83 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:09.615366 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24030c5f-dd1e-4639-8fb5-7fcc2df86f64 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:09.646357 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 791/1577] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:51:09 2022] POST /identity/v3/projects => generated 367 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:51:09.650244 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3225bd-579d-45c9-91da-a1bccfcf7c83 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:09.650354 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 787/1578] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:51:09 2022] POST /identity/v3/projects => generated 389 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:51:09.650930 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3303df-b3c4-49e4-97ed-4bb4917b2804 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:09.651291 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3303df-b3c4-49e4-97ed-4bb4917b2804 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:09.651894 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3303df-b3c4-49e4-97ed-4bb4917b2804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:09.670401 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-8d3303df-b3c4-49e4-97ed-4bb4917b2804 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:09.671720 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-8d3303df-b3c4-49e4-97ed-4bb4917b2804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:09.713642 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3225bd-579d-45c9-91da-a1bccfcf7c83 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:09.739726 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 788/1579] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:09.740589 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3303df-b3c4-49e4-97ed-4bb4917b2804 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:09.751060 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3303df-b3c4-49e4-97ed-4bb4917b2804 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:09.755747 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073c650a-b1b9-4c85-89ef-c18197705c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:09.756380 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c650a-b1b9-4c85-89ef-c18197705c1f admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:09.756660 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c650a-b1b9-4c85-89ef-c18197705c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:09.756937 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c650a-b1b9-4c85-89ef-c18197705c1f admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:09.758201 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073c650a-b1b9-4c85-89ef-c18197705c1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:09.760197 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073c650a-b1b9-4c85-89ef-c18197705c1f admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:09.771300 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:09.772324 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:09.781685 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d075bbe-9b66-4719-9d1e-4bd21962e85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:09.782559 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d075bbe-9b66-4719-9d1e-4bd21962e85c admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:09.783110 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d075bbe-9b66-4719-9d1e-4bd21962e85c admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:09.783583 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d075bbe-9b66-4719-9d1e-4bd21962e85c admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:09.787348 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 789/1580] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:51:09 2022] POST /identity/v3/users => generated 287 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:51:09.787508 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d075bbe-9b66-4719-9d1e-4bd21962e85c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:09.787625 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073c650a-b1b9-4c85-89ef-c18197705c1f admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:09.787753 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d075bbe-9b66-4719-9d1e-4bd21962e85c admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:09.795626 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073c650a-b1b9-4c85-89ef-c18197705c1f admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:09.802866 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f072b14-a31e-4a11-a626-4e46c1171504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:09.803898 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f072b14-a31e-4a11-a626-4e46c1171504 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:09.804693 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f072b14-a31e-4a11-a626-4e46c1171504 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:09.805104 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f072b14-a31e-4a11-a626-4e46c1171504 admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:09.805802 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f072b14-a31e-4a11-a626-4e46c1171504 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:09.807847 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f072b14-a31e-4a11-a626-4e46c1171504 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:09.812295 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 792/1581] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:51:09 2022] POST /identity/v3/users => generated 290 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:51:09.812295 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d075bbe-9b66-4719-9d1e-4bd21962e85c admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:09.820794 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d075bbe-9b66-4719-9d1e-4bd21962e85c admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:09.826068 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77422d58-294e-4e1e-905c-86364a5e3fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:09.826686 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77422d58-294e-4e1e-905c-86364a5e3fb4 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:09.826966 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77422d58-294e-4e1e-905c-86364a5e3fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:09.827276 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77422d58-294e-4e1e-905c-86364a5e3fb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:09.827985 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77422d58-294e-4e1e-905c-86364a5e3fb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:09.829596 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77422d58-294e-4e1e-905c-86364a5e3fb4 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:09.830592 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 790/1582] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:51:09 2022] POST /identity/v3/users => generated 289 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:51:09.831503 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f072b14-a31e-4a11-a626-4e46c1171504 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:09.842972 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f072b14-a31e-4a11-a626-4e46c1171504 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:09.843377 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 793/1583] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:51:09.844229 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77422d58-294e-4e1e-905c-86364a5e3fb4 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:09.848037 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9838316-7508-4308-ba77-24f6aef76375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:09.848662 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9838316-7508-4308-ba77-24f6aef76375 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:09.848973 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9838316-7508-4308-ba77-24f6aef76375 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:09.849296 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9838316-7508-4308-ba77-24f6aef76375 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:09.850108 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9838316-7508-4308-ba77-24f6aef76375 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:09.852372 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9838316-7508-4308-ba77-24f6aef76375 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:09.855480 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77422d58-294e-4e1e-905c-86364a5e3fb4 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:09.860334 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54923ebe-8705-4348-85df-1cb9b24a0aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:09.860943 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54923ebe-8705-4348-85df-1cb9b24a0aa9 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:09.861223 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54923ebe-8705-4348-85df-1cb9b24a0aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:09.861492 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54923ebe-8705-4348-85df-1cb9b24a0aa9 admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:09.862152 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54923ebe-8705-4348-85df-1cb9b24a0aa9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:09.864108 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54923ebe-8705-4348-85df-1cb9b24a0aa9 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:09.864219 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 791/1584] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:51:09.865170 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9838316-7508-4308-ba77-24f6aef76375 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:09.876516 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9838316-7508-4308-ba77-24f6aef76375 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:09.881177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473c102-8207-4306-a16d-057c41a9efb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:09.881803 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473c102-8207-4306-a16d-057c41a9efb8 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:09.882087 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473c102-8207-4306-a16d-057c41a9efb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:09.882931 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473c102-8207-4306-a16d-057c41a9efb8 admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:09.883610 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7473c102-8207-4306-a16d-057c41a9efb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:09.885632 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7473c102-8207-4306-a16d-057c41a9efb8 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:09.896243 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 794/1585] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:51:09 2022] POST /identity/v3/users => generated 284 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:51:09.900140 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54923ebe-8705-4348-85df-1cb9b24a0aa9 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:09.913678 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 792/1586] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:51:09 2022] POST /identity/v3/users => generated 295 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:51:09.913678 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473c102-8207-4306-a16d-057c41a9efb8 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:09.921707 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473c102-8207-4306-a16d-057c41a9efb8 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:09.926344 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd9b6e-57ec-4afa-a212-0486b1ed7c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:09.926958 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd9b6e-57ec-4afa-a212-0486b1ed7c29 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:09.927239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd9b6e-57ec-4afa-a212-0486b1ed7c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:09.927513 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd9b6e-57ec-4afa-a212-0486b1ed7c29 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:09.928253 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fd9b6e-57ec-4afa-a212-0486b1ed7c29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:09.930399 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fd9b6e-57ec-4afa-a212-0486b1ed7c29 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:09.940728 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 793/1587] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:51:09.941656 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd9b6e-57ec-4afa-a212-0486b1ed7c29 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:09.952915 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd9b6e-57ec-4afa-a212-0486b1ed7c29 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:09.958468 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee26c65-f267-4ea3-b9a1-08d4f0c9b77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:09.959095 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee26c65-f267-4ea3-b9a1-08d4f0c9b77f admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:09.959406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee26c65-f267-4ea3-b9a1-08d4f0c9b77f admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:09.959776 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee26c65-f267-4ea3-b9a1-08d4f0c9b77f admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:09.960604 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee26c65-f267-4ea3-b9a1-08d4f0c9b77f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:09.961915 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54923ebe-8705-4348-85df-1cb9b24a0aa9 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:09.962228 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee26c65-f267-4ea3-b9a1-08d4f0c9b77f admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:09.966402 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a621e1-c7ae-4261-931e-36dc5761add9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:09.967054 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a621e1-c7ae-4261-931e-36dc5761add9 service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:09.967330 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a621e1-c7ae-4261-931e-36dc5761add9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:09.967623 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a621e1-c7ae-4261-931e-36dc5761add9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:09.973221 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 794/1588] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:51:09.974141 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee26c65-f267-4ea3-b9a1-08d4f0c9b77f admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:09.977946 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a621e1-c7ae-4261-931e-36dc5761add9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:09.979430 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a621e1-c7ae-4261-931e-36dc5761add9 service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:09.984267 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee26c65-f267-4ea3-b9a1-08d4f0c9b77f admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:09.989548 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 795/1589] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:51:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:51:09.989681 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3921551-55da-4bbc-9f3f-542fe72939c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:09.990332 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a621e1-c7ae-4261-931e-36dc5761add9 service neutron] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:09.990458 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3921551-55da-4bbc-9f3f-542fe72939c6 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:09.990618 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3921551-55da-4bbc-9f3f-542fe72939c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:09.990889 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3921551-55da-4bbc-9f3f-542fe72939c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:09.992219 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3921551-55da-4bbc-9f3f-542fe72939c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:09.993831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3921551-55da-4bbc-9f3f-542fe72939c6 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:10.000739 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a621e1-c7ae-4261-931e-36dc5761add9 service neutron] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:10.005748 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 795/1590] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:51:10.006115 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ec2f7-b938-4640-a05b-240bce0b4af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:10.007056 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3921551-55da-4bbc-9f3f-542fe72939c6 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:10.007264 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ec2f7-b938-4640-a05b-240bce0b4af9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:10.007974 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ec2f7-b938-4640-a05b-240bce0b4af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:10.008251 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ec2f7-b938-4640-a05b-240bce0b4af9 admin admin] PATH_INFO: `/v3/projects/a0979ea73b614dcf9f120f5c713dd0fd/users/67de9c16b7a84740bbbfc63eab25c6e1/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:10.019742 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3921551-55da-4bbc-9f3f-542fe72939c6 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:10.025312 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0112eca5-ed8a-4f1e-b3d5-125c8c6d6d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:10.025473 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ec2f7-b938-4640-a05b-240bce0b4af9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0979ea73b614dcf9f120f5c713dd0fd, user_id=67de9c16b7a84740bbbfc63eab25c6e1, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:10.025935 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112eca5-ed8a-4f1e-b3d5-125c8c6d6d64 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:10.026221 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112eca5-ed8a-4f1e-b3d5-125c8c6d6d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:10.026497 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112eca5-ed8a-4f1e-b3d5-125c8c6d6d64 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:10.027197 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0112eca5-ed8a-4f1e-b3d5-125c8c6d6d64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:10.028361 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ec2f7-b938-4640-a05b-240bce0b4af9 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:10.028828 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0112eca5-ed8a-4f1e-b3d5-125c8c6d6d64 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:10.041526 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 796/1591] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:51:10.042556 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0112eca5-ed8a-4f1e-b3d5-125c8c6d6d64 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:10.051916 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 796/1592] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:51:09 2022] PUT /identity/v3/projects/a0979ea73b614dcf9f120f5c713dd0fd/users/67de9c16b7a84740bbbfc63eab25c6e1/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:10.054994 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ec2f7-b938-4640-a05b-240bce0b4af9 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:10.056558 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0112eca5-ed8a-4f1e-b3d5-125c8c6d6d64 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:10.127512 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25672cb-02e1-49c5-a701-bac97e787eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:10.128251 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25672cb-02e1-49c5-a701-bac97e787eab admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:10.128598 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25672cb-02e1-49c5-a701-bac97e787eab admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:10.128954 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25672cb-02e1-49c5-a701-bac97e787eab admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:10.129715 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25672cb-02e1-49c5-a701-bac97e787eab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:10.131423 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25672cb-02e1-49c5-a701-bac97e787eab admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:10.137126 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ec2f7-b938-4640-a05b-240bce0b4af9 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:10.142688 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 797/1593] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:10 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:51:10.143754 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25672cb-02e1-49c5-a701-bac97e787eab admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:10.146789 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d321de-5f5e-4b99-b1f5-9c64f1cda126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:10.147442 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d321de-5f5e-4b99-b1f5-9c64f1cda126 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:10.147746 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d321de-5f5e-4b99-b1f5-9c64f1cda126 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:10.148117 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d321de-5f5e-4b99-b1f5-9c64f1cda126 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:10.148827 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d321de-5f5e-4b99-b1f5-9c64f1cda126 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:10.150491 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d321de-5f5e-4b99-b1f5-9c64f1cda126 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:10.154970 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25672cb-02e1-49c5-a701-bac97e787eab admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:10.160258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c713332-4e06-4754-b6cc-849a3ece94f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:10.160957 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c713332-4e06-4754-b6cc-849a3ece94f7 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:10.161307 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c713332-4e06-4754-b6cc-849a3ece94f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:10.161646 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c713332-4e06-4754-b6cc-849a3ece94f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:10.162413 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c713332-4e06-4754-b6cc-849a3ece94f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:10.164894 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c713332-4e06-4754-b6cc-849a3ece94f7 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:10.165719 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 797/1594] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:10 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:51:10.167083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d321de-5f5e-4b99-b1f5-9c64f1cda126 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:10.182768 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 798/1595] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:51:10.183567 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c713332-4e06-4754-b6cc-849a3ece94f7 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:10.188960 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d321de-5f5e-4b99-b1f5-9c64f1cda126 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:10.197754 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c713332-4e06-4754-b6cc-849a3ece94f7 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:10.198800 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c4611-3a75-4512-a532-51e57a29ffc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:10.199548 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c4611-3a75-4512-a532-51e57a29ffc2 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:10.200014 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c4611-3a75-4512-a532-51e57a29ffc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:10.200369 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c4611-3a75-4512-a532-51e57a29ffc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:10.201258 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2c4611-3a75-4512-a532-51e57a29ffc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:10.202894 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2c4611-3a75-4512-a532-51e57a29ffc2 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:10.204516 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017ef8eb-f401-4960-bc02-973c8739880d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:10.205304 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ef8eb-f401-4960-bc02-973c8739880d admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:10.205747 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ef8eb-f401-4960-bc02-973c8739880d admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:10.206202 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ef8eb-f401-4960-bc02-973c8739880d admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:10.207050 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017ef8eb-f401-4960-bc02-973c8739880d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:10.208855 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017ef8eb-f401-4960-bc02-973c8739880d admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:10.216490 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 798/1596] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:10 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:51:10.220500 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c4611-3a75-4512-a532-51e57a29ffc2 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:10.220669 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 799/1597] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:51:10.222140 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b46b8-7dca-43b4-9381-470f9c69167f None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:10.222902 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b46b8-7dca-43b4-9381-470f9c69167f None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:10.222902 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b46b8-7dca-43b4-9381-470f9c69167f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:10.235047 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c4611-3a75-4512-a532-51e57a29ffc2 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:10.240443 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b1467-6f30-499b-a9ed-84661371da0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:10.241125 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b1467-6f30-499b-a9ed-84661371da0a admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:10.242509 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b1467-6f30-499b-a9ed-84661371da0a admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:10.242805 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b1467-6f30-499b-a9ed-84661371da0a admin admin] PATH_INFO: `/v3/projects/01fca015df86428b9356df3f127327c2/users/1401efb776ab424e8ff4f09eac33d92f/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:10.254006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-4b9b46b8-7dca-43b4-9381-470f9c69167f None None] MFA Rules not processed for user `67de9c16b7a84740bbbfc63eab25c6e1`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:10.255565 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-4b9b46b8-7dca-43b4-9381-470f9c69167f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:10.262116 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87b1467-6f30-499b-a9ed-84661371da0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=01fca015df86428b9356df3f127327c2, user_id=1401efb776ab424e8ff4f09eac33d92f, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:10.265294 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87b1467-6f30-499b-a9ed-84661371da0a admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:10.269365 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:10.288314 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 799/1598] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:51:10 2022] PUT /identity/v3/projects/01fca015df86428b9356df3f127327c2/users/1401efb776ab424e8ff4f09eac33d92f/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:10.289778 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b1467-6f30-499b-a9ed-84661371da0a admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:10.297817 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:10.351824 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 800/1599] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:10.355610 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b46b8-7dca-43b4-9381-470f9c69167f None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:10.363945 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b46b8-7dca-43b4-9381-470f9c69167f None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:10.377229 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b1467-6f30-499b-a9ed-84661371da0a admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:10.383672 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c770f016-9f66-4733-a52c-cfba8c65e7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:10.384439 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c770f016-9f66-4733-a52c-cfba8c65e7bc admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:10.384825 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c770f016-9f66-4733-a52c-cfba8c65e7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:10.385200 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c770f016-9f66-4733-a52c-cfba8c65e7bc admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:10.386014 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c770f016-9f66-4733-a52c-cfba8c65e7bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:10.387779 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c770f016-9f66-4733-a52c-cfba8c65e7bc admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:10.401403 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 800/1600] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:10 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:51:10.402393 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c770f016-9f66-4733-a52c-cfba8c65e7bc admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:10.427042 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c770f016-9f66-4733-a52c-cfba8c65e7bc admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:10.433817 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17135c4a-ec73-4e1a-af58-cb75a7f2a6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:10.434653 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17135c4a-ec73-4e1a-af58-cb75a7f2a6bd admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:10.435022 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17135c4a-ec73-4e1a-af58-cb75a7f2a6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:10.435403 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17135c4a-ec73-4e1a-af58-cb75a7f2a6bd admin admin] PATH_INFO: `/v3/projects/f70a0bbf5d2b49e99566672e689657ba/users/b515c2b9f1e94764ab6c3a6ddeaeb5c1/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:10.468373 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17135c4a-ec73-4e1a-af58-cb75a7f2a6bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f70a0bbf5d2b49e99566672e689657ba, user_id=b515c2b9f1e94764ab6c3a6ddeaeb5c1, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:10.471464 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17135c4a-ec73-4e1a-af58-cb75a7f2a6bd admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:10.472042 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599ecc28-064d-4c22-8865-a54430aaac89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:10.474318 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ecc28-064d-4c22-8865-a54430aaac89 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:10.475397 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ecc28-064d-4c22-8865-a54430aaac89 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:10.476231 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ecc28-064d-4c22-8865-a54430aaac89 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:10.477782 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599ecc28-064d-4c22-8865-a54430aaac89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:10.481910 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599ecc28-064d-4c22-8865-a54430aaac89 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:10.487856 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 801/1601] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:51:10 2022] PUT /identity/v3/projects/f70a0bbf5d2b49e99566672e689657ba/users/b515c2b9f1e94764ab6c3a6ddeaeb5c1/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:10.489058 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17135c4a-ec73-4e1a-af58-cb75a7f2a6bd admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:10.509123 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 801/1602] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:10 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:51:10.511294 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599ecc28-064d-4c22-8865-a54430aaac89 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:10.537663 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599ecc28-064d-4c22-8865-a54430aaac89 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:10.589040 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17135c4a-ec73-4e1a-af58-cb75a7f2a6bd admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:10.594076 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f42c2bb-c74f-4aa9-97b2-abd92395ae0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:10.594962 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f42c2bb-c74f-4aa9-97b2-abd92395ae0a admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:10.595310 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f42c2bb-c74f-4aa9-97b2-abd92395ae0a admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:10.595702 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f42c2bb-c74f-4aa9-97b2-abd92395ae0a admin admin] PATH_INFO: `/v3/projects/0f35a4b7696f4cdfbb6e6b316352c022/users/7c4ed2f2f6994ae2bb00d5a502715b4f/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:10.614199 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f42c2bb-c74f-4aa9-97b2-abd92395ae0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f35a4b7696f4cdfbb6e6b316352c022, user_id=7c4ed2f2f6994ae2bb00d5a502715b4f, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:10.615886 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a279d011-937b-4c12-b305-a1214a68eb54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:10.617427 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279d011-937b-4c12-b305-a1214a68eb54 admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:10.617427 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279d011-937b-4c12-b305-a1214a68eb54 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:10.617863 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279d011-937b-4c12-b305-a1214a68eb54 admin admin] PATH_INFO: `/v3/projects/2978eaf529f540eab10fde695e9f049d/users/9a0faf8c26394a5b9d5c7e23756a19ea/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:10.618202 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f42c2bb-c74f-4aa9-97b2-abd92395ae0a admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:10.640406 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 802/1603] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:51:10 2022] PUT /identity/v3/projects/0f35a4b7696f4cdfbb6e6b316352c022/users/7c4ed2f2f6994ae2bb00d5a502715b4f/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:10.640610 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a279d011-937b-4c12-b305-a1214a68eb54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2978eaf529f540eab10fde695e9f049d, user_id=9a0faf8c26394a5b9d5c7e23756a19ea, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:10.640712 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986fdf7d-7b0c-4bd0-8de8-44a4d8e26060 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:10.640712 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986fdf7d-7b0c-4bd0-8de8-44a4d8e26060 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:10.640873 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: WARNING py.warnings [None req-a279d011-937b-4c12-b305-a1214a68eb54 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 12:51:10.640873 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: warnings.warn(msg) Jul 26 12:51:10.640873 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: Jul 26 12:51:10.641179 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986fdf7d-7b0c-4bd0-8de8-44a4d8e26060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:10.641981 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a279d011-937b-4c12-b305-a1214a68eb54 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:10.658543 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 802/1604] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:51:10 2022] PUT /identity/v3/projects/2978eaf529f540eab10fde695e9f049d/users/9a0faf8c26394a5b9d5c7e23756a19ea/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:10.659004 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a279d011-937b-4c12-b305-a1214a68eb54 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:10.665629 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-986fdf7d-7b0c-4bd0-8de8-44a4d8e26060 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:10.668263 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-986fdf7d-7b0c-4bd0-8de8-44a4d8e26060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:10.716294 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:10.728585 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:10.738651 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a279d011-937b-4c12-b305-a1214a68eb54 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:10.743928 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d21625-3ffb-4eb6-b302-048a86483c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:10.744558 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d21625-3ffb-4eb6-b302-048a86483c72 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:10.744847 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d21625-3ffb-4eb6-b302-048a86483c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:10.745126 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d21625-3ffb-4eb6-b302-048a86483c72 admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:10.745998 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d21625-3ffb-4eb6-b302-048a86483c72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:10.746098 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 803/1605] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:10.746851 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986fdf7d-7b0c-4bd0-8de8-44a4d8e26060 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:10.753058 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d21625-3ffb-4eb6-b302-048a86483c72 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:10.761141 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986fdf7d-7b0c-4bd0-8de8-44a4d8e26060 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:10.767709 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f584bb8a-ec60-412c-95bc-9e3a438c2f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:10.768484 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584bb8a-ec60-412c-95bc-9e3a438c2f91 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:10.768819 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584bb8a-ec60-412c-95bc-9e3a438c2f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:10.769256 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584bb8a-ec60-412c-95bc-9e3a438c2f91 admin admin] PATH_INFO: `/v3/projects/65cb1e6051ad4601870e2d74ce55ce44/users/d9a3980b6df44adb879f3507ca9167da/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:10.789836 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f584bb8a-ec60-412c-95bc-9e3a438c2f91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65cb1e6051ad4601870e2d74ce55ce44, user_id=d9a3980b6df44adb879f3507ca9167da, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:10.793887 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f584bb8a-ec60-412c-95bc-9e3a438c2f91 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:10.803353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 803/1606] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:51:10 2022] POST /identity/v3/projects => generated 373 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:51:10.805271 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a5c96-1282-450e-94c1-6de781662aed None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:10.805616 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a5c96-1282-450e-94c1-6de781662aed None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:10.805947 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a5c96-1282-450e-94c1-6de781662aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:10.813361 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 804/1607] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:51:10 2022] PUT /identity/v3/projects/65cb1e6051ad4601870e2d74ce55ce44/users/d9a3980b6df44adb879f3507ca9167da/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:10.813995 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f584bb8a-ec60-412c-95bc-9e3a438c2f91 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:10.860766 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-5d5a5c96-1282-450e-94c1-6de781662aed None None] MFA Rules not processed for user `b515c2b9f1e94764ab6c3a6ddeaeb5c1`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:10.862589 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-5d5a5c96-1282-450e-94c1-6de781662aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:10.915202 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f584bb8a-ec60-412c-95bc-9e3a438c2f91 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:10.923892 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade13ba-bd81-403c-83b7-095d05ccc261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:10.925253 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade13ba-bd81-403c-83b7-095d05ccc261 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:10.925876 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade13ba-bd81-403c-83b7-095d05ccc261 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:10.927981 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade13ba-bd81-403c-83b7-095d05ccc261 admin admin] PATH_INFO: `/v3/projects/a93dd858fb7a40fbbb679fbe676b1f3c/users/17b7e2c1b1b04366b99e9ab892aeb835/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:10.953057 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade13ba-bd81-403c-83b7-095d05ccc261 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a93dd858fb7a40fbbb679fbe676b1f3c, user_id=17b7e2c1b1b04366b99e9ab892aeb835, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:10.955319 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 804/1608] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:10.956685 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918afde-31c7-43ae-9daf-788b99dbd55b None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:10.957067 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918afde-31c7-43ae-9daf-788b99dbd55b None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:10.957437 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918afde-31c7-43ae-9daf-788b99dbd55b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:10.957900 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade13ba-bd81-403c-83b7-095d05ccc261 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:10.982027 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-3918afde-31c7-43ae-9daf-788b99dbd55b None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:10.983525 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-3918afde-31c7-43ae-9daf-788b99dbd55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:10.999111 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 805/1609] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:51:10 2022] PUT /identity/v3/projects/a93dd858fb7a40fbbb679fbe676b1f3c/users/17b7e2c1b1b04366b99e9ab892aeb835/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:11.001608 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db560f-27a6-4c54-a659-4f58e1cee61d None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:11.001903 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db560f-27a6-4c54-a659-4f58e1cee61d None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:11.003813 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db560f-27a6-4c54-a659-4f58e1cee61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:11.058968 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 805/1610] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:11.058968 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3918afde-31c7-43ae-9daf-788b99dbd55b None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:11.060492 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-87db560f-27a6-4c54-a659-4f58e1cee61d None None] MFA Rules not processed for user `9a0faf8c26394a5b9d5c7e23756a19ea`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:11.062033 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-87db560f-27a6-4c54-a659-4f58e1cee61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:11.125279 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3918afde-31c7-43ae-9daf-788b99dbd55b None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:11.130996 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b74edd-73c0-40f8-98b5-fb596ee89699 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:11.132013 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b74edd-73c0-40f8-98b5-fb596ee89699 service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:11.132930 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b74edd-73c0-40f8-98b5-fb596ee89699 service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:11.133307 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b74edd-73c0-40f8-98b5-fb596ee89699 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:11.145620 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b74edd-73c0-40f8-98b5-fb596ee89699 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:11.147237 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b74edd-73c0-40f8-98b5-fb596ee89699 service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:11.162082 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 806/1611] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:51:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:51:11.163064 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b74edd-73c0-40f8-98b5-fb596ee89699 service neutron] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:11.174291 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b74edd-73c0-40f8-98b5-fb596ee89699 service neutron] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:11.179502 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 806/1612] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:11.180835 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cef5a2-b088-4b12-9608-37e7ebda03f2 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:11.181141 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cef5a2-b088-4b12-9608-37e7ebda03f2 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:11.181395 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cef5a2-b088-4b12-9608-37e7ebda03f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:11.192264 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f99247-5623-4395-a0bf-0b6922114126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:11.193181 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f99247-5623-4395-a0bf-0b6922114126 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:11.193813 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f99247-5623-4395-a0bf-0b6922114126 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:11.193813 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f99247-5623-4395-a0bf-0b6922114126 admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:11.194566 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f99247-5623-4395-a0bf-0b6922114126 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:11.197784 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f99247-5623-4395-a0bf-0b6922114126 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:11.206625 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:11.224721 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 807/1613] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:51:11 2022] POST /identity/v3/users => generated 293 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:51:11.227165 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2722b43-b2a5-4271-8ecb-61835a932f8e None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:11.227165 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2722b43-b2a5-4271-8ecb-61835a932f8e None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:11.227395 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2722b43-b2a5-4271-8ecb-61835a932f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:11.232962 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-18cef5a2-b088-4b12-9608-37e7ebda03f2 None None] MFA Rules not processed for user `d9a3980b6df44adb879f3507ca9167da`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:11.235067 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-18cef5a2-b088-4b12-9608-37e7ebda03f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:11.250355 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-f2722b43-b2a5-4271-8ecb-61835a932f8e None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:11.251775 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-f2722b43-b2a5-4271-8ecb-61835a932f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:11.261107 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 808/1614] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:11.263007 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce822e-7333-42a8-a45c-b4f46d58fbe9 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:11.263252 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce822e-7333-42a8-a45c-b4f46d58fbe9 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:11.269523 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce822e-7333-42a8-a45c-b4f46d58fbe9 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:11.271192 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 809/1615] 10.222.0.20 () {54 vars in 962 bytes} [Tue Jul 26 12:51:11 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:51:11.278317 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce822e-7333-42a8-a45c-b4f46d58fbe9 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:11.280926 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-07ce822e-7333-42a8-a45c-b4f46d58fbe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:11.281544 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:11.298117 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce822e-7333-42a8-a45c-b4f46d58fbe9 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:11.301983 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f96b0-9f2a-47ee-b0c5-bbfac8fd0cea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:11.302723 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f96b0-9f2a-47ee-b0c5-bbfac8fd0cea service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:11.303130 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f96b0-9f2a-47ee-b0c5-bbfac8fd0cea service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:11.303503 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f96b0-9f2a-47ee-b0c5-bbfac8fd0cea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:11.315907 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929f96b0-9f2a-47ee-b0c5-bbfac8fd0cea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:11.317677 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929f96b0-9f2a-47ee-b0c5-bbfac8fd0cea service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:11.339887 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 810/1616] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:51:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:51:11.341101 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f96b0-9f2a-47ee-b0c5-bbfac8fd0cea service neutron] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:11.345604 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 807/1617] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:11 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:11.347246 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cef5a2-b088-4b12-9608-37e7ebda03f2 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:11.362012 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cef5a2-b088-4b12-9608-37e7ebda03f2 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:11.365196 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f96b0-9f2a-47ee-b0c5-bbfac8fd0cea service neutron] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:11.370131 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff4a269-a4a1-4688-aac9-2e72547891f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:11.370987 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4a269-a4a1-4688-aac9-2e72547891f1 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:11.371279 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4a269-a4a1-4688-aac9-2e72547891f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:11.371685 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4a269-a4a1-4688-aac9-2e72547891f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:11.372796 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff4a269-a4a1-4688-aac9-2e72547891f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:11.374917 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff4a269-a4a1-4688-aac9-2e72547891f1 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:11.378271 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc22c09-18a7-444c-8d09-91a72271987a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:11.379168 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc22c09-18a7-444c-8d09-91a72271987a admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:11.379594 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc22c09-18a7-444c-8d09-91a72271987a admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:11.383055 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc22c09-18a7-444c-8d09-91a72271987a admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:11.384492 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc22c09-18a7-444c-8d09-91a72271987a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:11.388307 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc22c09-18a7-444c-8d09-91a72271987a admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:11.391522 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 808/1618] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:51:11.392777 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff4a269-a4a1-4688-aac9-2e72547891f1 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:11.406319 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff4a269-a4a1-4688-aac9-2e72547891f1 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:11.411831 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6648ea0c-fe3b-4d04-9bf7-00cc75f614ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:11.412802 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648ea0c-fe3b-4d04-9bf7-00cc75f614ff service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:11.413232 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648ea0c-fe3b-4d04-9bf7-00cc75f614ff service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:11.413767 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648ea0c-fe3b-4d04-9bf7-00cc75f614ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:11.425742 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6648ea0c-fe3b-4d04-9bf7-00cc75f614ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:11.427994 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6648ea0c-fe3b-4d04-9bf7-00cc75f614ff service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:11.447159 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 809/1619] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:51:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:51:11.449107 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a80541-d8c8-4746-aabf-9b8f15b03e6b None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:11.449743 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a80541-d8c8-4746-aabf-9b8f15b03e6b None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:11.450344 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a80541-d8c8-4746-aabf-9b8f15b03e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:11.458648 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 811/1620] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:51:11 2022] POST /identity/v3/projects => generated 375 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:51:11.462476 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc22c09-18a7-444c-8d09-91a72271987a admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:11.516580 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-95a80541-d8c8-4746-aabf-9b8f15b03e6b None None] MFA Rules not processed for user `0f5d6f3c12244c0faf26048513dc49e7`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:11.520893 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-95a80541-d8c8-4746-aabf-9b8f15b03e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:11.609776 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc22c09-18a7-444c-8d09-91a72271987a admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:11.617501 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b035ddc2-3fdc-4f03-8c8c-550fae9161ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:11.618528 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b035ddc2-3fdc-4f03-8c8c-550fae9161ea service cinder] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:11.619029 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b035ddc2-3fdc-4f03-8c8c-550fae9161ea service cinder] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:11.619530 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b035ddc2-3fdc-4f03-8c8c-550fae9161ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:11.701428 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 810/1621] 10.222.0.20 () {58 vars in 1059 bytes} [Tue Jul 26 12:51:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:11.714411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a80541-d8c8-4746-aabf-9b8f15b03e6b None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:11.747515 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b035ddc2-3fdc-4f03-8c8c-550fae9161ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:11.749221 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b035ddc2-3fdc-4f03-8c8c-550fae9161ea service cinder] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:11.762397 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 812/1622] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:51:11 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:51:11.763619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b035ddc2-3fdc-4f03-8c8c-550fae9161ea service cinder] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:11.855591 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a80541-d8c8-4746-aabf-9b8f15b03e6b None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:11.866178 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e552da19-aab1-4fcd-ab8b-d92c0403e39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:11.867051 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e552da19-aab1-4fcd-ab8b-d92c0403e39b admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:11.867447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e552da19-aab1-4fcd-ab8b-d92c0403e39b admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:11.867861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e552da19-aab1-4fcd-ab8b-d92c0403e39b admin admin] PATH_INFO: `/v3/projects/f6a99a83ae1743789d46512f0786c1c8/users/3fdbd72d7ad74c9bb0a8c81c3b921333/roles/0e78444e43c744668178971af4087ca0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:11.905768 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b035ddc2-3fdc-4f03-8c8c-550fae9161ea service cinder] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:11.909831 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e552da19-aab1-4fcd-ab8b-d92c0403e39b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6a99a83ae1743789d46512f0786c1c8, user_id=3fdbd72d7ad74c9bb0a8c81c3b921333, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:11.912882 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e552da19-aab1-4fcd-ab8b-d92c0403e39b admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:11.919279 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257a8930-fce1-4063-8016-64b0df69670f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:11.920069 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a8930-fce1-4063-8016-64b0df69670f admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:11.920316 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a8930-fce1-4063-8016-64b0df69670f admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:11.920573 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a8930-fce1-4063-8016-64b0df69670f admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:11.924573 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257a8930-fce1-4063-8016-64b0df69670f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:11.929278 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257a8930-fce1-4063-8016-64b0df69670f admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:11.935627 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 811/1623] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:51:11 2022] PUT /identity/v3/projects/f6a99a83ae1743789d46512f0786c1c8/users/3fdbd72d7ad74c9bb0a8c81c3b921333/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:11.936902 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e552da19-aab1-4fcd-ab8b-d92c0403e39b admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:11.955085 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:11.957842 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:11.986914 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 813/1624] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:51:11 2022] POST /identity/v3/users => generated 294 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:51:11.988847 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257a8930-fce1-4063-8016-64b0df69670f admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:12.021942 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257a8930-fce1-4063-8016-64b0df69670f admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:12.137295 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e552da19-aab1-4fcd-ab8b-d92c0403e39b admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:12.152950 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48208b8-efd0-4317-843e-a856404b84d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:12.153598 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48208b8-efd0-4317-843e-a856404b84d5 service nova] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:12.154007 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48208b8-efd0-4317-843e-a856404b84d5 service nova] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:12.154296 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48208b8-efd0-4317-843e-a856404b84d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:12.181910 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419f4922-1b0e-4722-9047-508ea1c8773a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:12.182980 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f4922-1b0e-4722-9047-508ea1c8773a admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:12.183844 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f4922-1b0e-4722-9047-508ea1c8773a admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:12.185554 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f4922-1b0e-4722-9047-508ea1c8773a admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:12.186579 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419f4922-1b0e-4722-9047-508ea1c8773a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:12.191323 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419f4922-1b0e-4722-9047-508ea1c8773a admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:12.231607 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 814/1625] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:51:12.232901 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419f4922-1b0e-4722-9047-508ea1c8773a admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:12.251622 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419f4922-1b0e-4722-9047-508ea1c8773a admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:12.259191 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21cb54b-ea91-4cc6-a39d-be661f212d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:12.260124 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21cb54b-ea91-4cc6-a39d-be661f212d1f admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:12.260522 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21cb54b-ea91-4cc6-a39d-be661f212d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:12.261217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21cb54b-ea91-4cc6-a39d-be661f212d1f admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:12.264577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21cb54b-ea91-4cc6-a39d-be661f212d1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:12.271840 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21cb54b-ea91-4cc6-a39d-be661f212d1f admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:12.295415 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48208b8-efd0-4317-843e-a856404b84d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:12.302127 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 815/1626] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:12 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:51:12.303731 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21cb54b-ea91-4cc6-a39d-be661f212d1f admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:12.306921 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48208b8-efd0-4317-843e-a856404b84d5 service nova] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:12.319070 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21cb54b-ea91-4cc6-a39d-be661f212d1f admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:12.329438 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a9f2ed-0c0a-433b-9c99-b70629735da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:12.330977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9f2ed-0c0a-433b-9c99-b70629735da2 admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:12.331402 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9f2ed-0c0a-433b-9c99-b70629735da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:12.331921 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9f2ed-0c0a-433b-9c99-b70629735da2 admin admin] PATH_INFO: `/v3/projects/f6a99a83ae1743789d46512f0786c1c8/users/3fdbd72d7ad74c9bb0a8c81c3b921333/roles/0e78444e43c744668178971af4087ca0` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:12.339150 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a9f2ed-0c0a-433b-9c99-b70629735da2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6a99a83ae1743789d46512f0786c1c8, user_id=3fdbd72d7ad74c9bb0a8c81c3b921333, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:12.344114 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a9f2ed-0c0a-433b-9c99-b70629735da2 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:12.357244 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 812/1627] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:51:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:51:12.359437 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48208b8-efd0-4317-843e-a856404b84d5 service nova] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:12.373897 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 816/1628] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:51:12 2022] PUT /identity/v3/projects/f6a99a83ae1743789d46512f0786c1c8/users/3fdbd72d7ad74c9bb0a8c81c3b921333/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:12.375906 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c93964-ade2-4cc5-894d-1deaa318d766 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:12.376306 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c93964-ade2-4cc5-894d-1deaa318d766 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:12.376822 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c93964-ade2-4cc5-894d-1deaa318d766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:12.488382 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48208b8-efd0-4317-843e-a856404b84d5 service nova] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:12.491016 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-b2c93964-ade2-4cc5-894d-1deaa318d766 None None] MFA Rules not processed for user `3fdbd72d7ad74c9bb0a8c81c3b921333`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:12.491016 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-b2c93964-ade2-4cc5-894d-1deaa318d766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:12.592403 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3318485-c69c-4566-887f-2dd3e68ab849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:12.593793 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3318485-c69c-4566-887f-2dd3e68ab849 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:12.593994 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3318485-c69c-4566-887f-2dd3e68ab849 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:12.594246 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3318485-c69c-4566-887f-2dd3e68ab849 admin admin] PATH_INFO: `/v3/projects/9e4e02d8829643d99aaaa3d055c4df9a/users/36d31fe235d749d28f251aed98952280/roles/0e78444e43c744668178971af4087ca0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:12.631119 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3318485-c69c-4566-887f-2dd3e68ab849 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e4e02d8829643d99aaaa3d055c4df9a, user_id=36d31fe235d749d28f251aed98952280, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:12.635026 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3318485-c69c-4566-887f-2dd3e68ab849 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:12.660782 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 813/1629] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:51:12 2022] PUT /identity/v3/projects/9e4e02d8829643d99aaaa3d055c4df9a/users/36d31fe235d749d28f251aed98952280/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:12.662361 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3318485-c69c-4566-887f-2dd3e68ab849 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:12.682685 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 817/1630] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:12 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:12.695138 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c93964-ade2-4cc5-894d-1deaa318d766 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:12.718294 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:12.778970 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3318485-c69c-4566-887f-2dd3e68ab849 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:12.786327 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0004d7-4978-4dcc-8468-ce2e9e9b5b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:12.787247 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0004d7-4978-4dcc-8468-ce2e9e9b5b1f admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:12.787750 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0004d7-4978-4dcc-8468-ce2e9e9b5b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:12.788238 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0004d7-4978-4dcc-8468-ce2e9e9b5b1f admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:12.789750 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0004d7-4978-4dcc-8468-ce2e9e9b5b1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:12.791986 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0004d7-4978-4dcc-8468-ce2e9e9b5b1f admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:12.798311 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:12.809140 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c93964-ade2-4cc5-894d-1deaa318d766 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:12.852646 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 814/1631] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:12 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:51:12.854255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0004d7-4978-4dcc-8468-ce2e9e9b5b1f admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:12.888175 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0004d7-4978-4dcc-8468-ce2e9e9b5b1f admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:12.897332 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded9c07-221f-43d6-a715-2aa84a72ed66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:12.898718 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded9c07-221f-43d6-a715-2aa84a72ed66 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:12.899071 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded9c07-221f-43d6-a715-2aa84a72ed66 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:12.900057 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded9c07-221f-43d6-a715-2aa84a72ed66 admin admin] PATH_INFO: `/v3/projects/9e4e02d8829643d99aaaa3d055c4df9a/users/36d31fe235d749d28f251aed98952280/roles/0e78444e43c744668178971af4087ca0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:12.903304 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded9c07-221f-43d6-a715-2aa84a72ed66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e4e02d8829643d99aaaa3d055c4df9a, user_id=36d31fe235d749d28f251aed98952280, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:12.908611 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded9c07-221f-43d6-a715-2aa84a72ed66 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:12.916394 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a3eddd-cc6e-4cbd-89d2-4817e67e23ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:12.917038 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3eddd-cc6e-4cbd-89d2-4817e67e23ee service cinder] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:12.917735 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3eddd-cc6e-4cbd-89d2-4817e67e23ee service cinder] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:12.918018 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3eddd-cc6e-4cbd-89d2-4817e67e23ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:12.935743 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 815/1632] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:51:12 2022] PUT /identity/v3/projects/9e4e02d8829643d99aaaa3d055c4df9a/users/36d31fe235d749d28f251aed98952280/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:12.956609 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c246066-dace-4f7f-979a-53cd97bc6cb1 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:12.958121 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c246066-dace-4f7f-979a-53cd97bc6cb1 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:12.959020 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c246066-dace-4f7f-979a-53cd97bc6cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:13.045837 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-7c246066-dace-4f7f-979a-53cd97bc6cb1 None None] MFA Rules not processed for user `36d31fe235d749d28f251aed98952280`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:13.047069 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-7c246066-dace-4f7f-979a-53cd97bc6cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:13.090581 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a3eddd-cc6e-4cbd-89d2-4817e67e23ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:13.098291 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a3eddd-cc6e-4cbd-89d2-4817e67e23ee service cinder] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:13.151489 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 818/1633] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:51:12 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 12:51:13.308727 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 816/1634] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:12 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:51:13.333107 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a3eddd-cc6e-4cbd-89d2-4817e67e23ee service cinder] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:13.361550 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a3eddd-cc6e-4cbd-89d2-4817e67e23ee service cinder] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:13.450669 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c246066-dace-4f7f-979a-53cd97bc6cb1 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:13.544836 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b167efab-72cf-4c8c-8a6c-210c3795a41d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:13.545655 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167efab-72cf-4c8c-8a6c-210c3795a41d service cinder] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:13.546048 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167efab-72cf-4c8c-8a6c-210c3795a41d service cinder] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:13.548610 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167efab-72cf-4c8c-8a6c-210c3795a41d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:13.572287 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c246066-dace-4f7f-979a-53cd97bc6cb1 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:13.586928 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9fc0ab-b2bc-484c-9981-7bcec3413f6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:13.595801 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9fc0ab-b2bc-484c-9981-7bcec3413f6d service cinder] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:13.596138 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9fc0ab-b2bc-484c-9981-7bcec3413f6d service cinder] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:13.596421 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9fc0ab-b2bc-484c-9981-7bcec3413f6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:13.672748 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b167efab-72cf-4c8c-8a6c-210c3795a41d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:13.674226 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b167efab-72cf-4c8c-8a6c-210c3795a41d service cinder] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:13.689545 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 819/1635] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:51:13 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:51:13.691231 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2faeca-4e67-449c-bbfa-33747518df7f None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:13.691527 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2faeca-4e67-449c-bbfa-33747518df7f None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:13.692622 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2faeca-4e67-449c-bbfa-33747518df7f None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:13.693975 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 820/1636] 10.222.0.20 () {54 vars in 962 bytes} [Tue Jul 26 12:51:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:51:13.699973 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71133bb7-8e38-44ef-92d1-9ec425a0710c None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:13.700529 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71133bb7-8e38-44ef-92d1-9ec425a0710c None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:13.700810 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71133bb7-8e38-44ef-92d1-9ec425a0710c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:13.706045 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:13.730731 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9fc0ab-b2bc-484c-9981-7bcec3413f6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:13.732252 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9fc0ab-b2bc-484c-9981-7bcec3413f6d service cinder] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:13.743759 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-71133bb7-8e38-44ef-92d1-9ec425a0710c None None] MFA Rules not processed for user `0f5d6f3c12244c0faf26048513dc49e7`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:13.746191 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-71133bb7-8e38-44ef-92d1-9ec425a0710c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:13.754565 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 817/1637] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:51:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 12:51:13.754997 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:13.840235 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 821/1638] 10.222.0.20 () {58 vars in 1059 bytes} [Tue Jul 26 12:51:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:51:13.846192 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62406f2f-df10-4da2-951c-0dff83ce8548 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:13.847307 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62406f2f-df10-4da2-951c-0dff83ce8548 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:13.848316 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62406f2f-df10-4da2-951c-0dff83ce8548 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:13.850111 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 818/1639] 10.222.0.20 () {54 vars in 912 bytes} [Tue Jul 26 12:51:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:51:13.856777 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804bf053-618c-4e79-88a3-27ababbb3481 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:13.857903 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804bf053-618c-4e79-88a3-27ababbb3481 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:13.858451 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804bf053-618c-4e79-88a3-27ababbb3481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:13.907498 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-804bf053-618c-4e79-88a3-27ababbb3481 None None] MFA Rules not processed for user `0f5d6f3c12244c0faf26048513dc49e7`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:13.914446 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-804bf053-618c-4e79-88a3-27ababbb3481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:13.937616 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 822/1640] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Jul 26 12:51:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:13.989156 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62406f2f-df10-4da2-951c-0dff83ce8548 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:14.004560 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62406f2f-df10-4da2-951c-0dff83ce8548 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:14.009376 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e271f1f4-a75a-4ac6-9f1f-1590178cd0ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:14.010112 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271f1f4-a75a-4ac6-9f1f-1590178cd0ec service cinder] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:14.010463 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271f1f4-a75a-4ac6-9f1f-1590178cd0ec service cinder] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:14.010978 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271f1f4-a75a-4ac6-9f1f-1590178cd0ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:14.026310 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e271f1f4-a75a-4ac6-9f1f-1590178cd0ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:14.028289 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e271f1f4-a75a-4ac6-9f1f-1590178cd0ec service cinder] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:14.049319 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 819/1641] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:51:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:51:14.200347 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0c871-6be9-4131-bca0-1b3e33566dd7 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:14.200578 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0c871-6be9-4131-bca0-1b3e33566dd7 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:14.201344 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0c871-6be9-4131-bca0-1b3e33566dd7 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:14.203340 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 823/1642] 10.222.0.20 () {54 vars in 912 bytes} [Tue Jul 26 12:51:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:51:14.208869 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5bbbe1-42cd-4151-a670-24e44f9656d4 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:14.209301 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5bbbe1-42cd-4151-a670-24e44f9656d4 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:14.209680 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5bbbe1-42cd-4151-a670-24e44f9656d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:14.251723 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-fa5bbbe1-42cd-4151-a670-24e44f9656d4 None None] MFA Rules not processed for user `0f5d6f3c12244c0faf26048513dc49e7`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:14.256589 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-fa5bbbe1-42cd-4151-a670-24e44f9656d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:14.278802 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 820/1643] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Jul 26 12:51:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:14.342329 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b0c871-6be9-4131-bca0-1b3e33566dd7 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:14.356944 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b0c871-6be9-4131-bca0-1b3e33566dd7 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:14.362190 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9fcf1d-ede8-4355-83e8-695afa92dff9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:14.363078 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9fcf1d-ede8-4355-83e8-695afa92dff9 service cinder] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:14.363669 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9fcf1d-ede8-4355-83e8-695afa92dff9 service cinder] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:14.364236 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9fcf1d-ede8-4355-83e8-695afa92dff9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:14.382440 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9fcf1d-ede8-4355-83e8-695afa92dff9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:14.385592 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9fcf1d-ede8-4355-83e8-695afa92dff9 service cinder] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:14.416828 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 824/1644] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:51:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:51:14.640977 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5bbbe1-42cd-4151-a670-24e44f9656d4 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:14.811123 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5bbbe1-42cd-4151-a670-24e44f9656d4 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:14.821972 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d847b1-5219-47fd-98fe-2e45dd4d56f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:14.823957 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d847b1-5219-47fd-98fe-2e45dd4d56f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:14.824511 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d847b1-5219-47fd-98fe-2e45dd4d56f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:14.825485 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d847b1-5219-47fd-98fe-2e45dd4d56f7 admin admin] PATH_INFO: `/v3/users/b515c2b9f1e94764ab6c3a6ddeaeb5c1` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:14.830831 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d847b1-5219-47fd-98fe-2e45dd4d56f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b515c2b9f1e94764ab6c3a6ddeaeb5c1)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:14.837615 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d847b1-5219-47fd-98fe-2e45dd4d56f7 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:14.968776 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-30d847b1-5219-47fd-98fe-2e45dd4d56f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b515c2b9f1e94764ab6c3a6ddeaeb5c1'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:14.970411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-30d847b1-5219-47fd-98fe-2e45dd4d56f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b515c2b9f1e94764ab6c3a6ddeaeb5c1'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:14.975189 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-30d847b1-5219-47fd-98fe-2e45dd4d56f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b515c2b9f1e94764ab6c3a6ddeaeb5c1'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:15.018802 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-30d847b1-5219-47fd-98fe-2e45dd4d56f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b515c2b9f1e94764ab6c3a6ddeaeb5c1'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:15.052167 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-30d847b1-5219-47fd-98fe-2e45dd4d56f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b515c2b9f1e94764ab6c3a6ddeaeb5c1'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:15.088280 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 821/1645] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:51:14 2022] DELETE /identity/v3/users/b515c2b9f1e94764ab6c3a6ddeaeb5c1 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:15.093792 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33ba5a-3e17-4237-8098-f1d0ef229b26 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:15.094288 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33ba5a-3e17-4237-8098-f1d0ef229b26 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:15.094288 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33ba5a-3e17-4237-8098-f1d0ef229b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:15.120695 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-2c33ba5a-3e17-4237-8098-f1d0ef229b26 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:15.122242 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-2c33ba5a-3e17-4237-8098-f1d0ef229b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:15.286558 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 825/1646] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:15.405119 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d847b1-5219-47fd-98fe-2e45dd4d56f7 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:15.406900 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-30d847b1-5219-47fd-98fe-2e45dd4d56f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:15.622906 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:15.638341 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d847b1-5219-47fd-98fe-2e45dd4d56f7 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:15.645451 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f906c-2a2b-4e67-93b8-31a6b6669f99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:15.647023 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f906c-2a2b-4e67-93b8-31a6b6669f99 service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:15.647309 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f906c-2a2b-4e67-93b8-31a6b6669f99 service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:15.649992 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f906c-2a2b-4e67-93b8-31a6b6669f99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:15.678636 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3f906c-2a2b-4e67-93b8-31a6b6669f99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:15.685625 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3f906c-2a2b-4e67-93b8-31a6b6669f99 service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:15.713919 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 822/1647] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:51:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:51:16.794836 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c33ba5a-3e17-4237-8098-f1d0ef229b26 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:16.796796 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-2c33ba5a-3e17-4237-8098-f1d0ef229b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:16.821445 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c33ba5a-3e17-4237-8098-f1d0ef229b26 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:16.836552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f5a4de-87d6-4f82-8ecf-1af3b4220f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:16.837396 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5a4de-87d6-4f82-8ecf-1af3b4220f6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:16.837852 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5a4de-87d6-4f82-8ecf-1af3b4220f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:16.839739 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5a4de-87d6-4f82-8ecf-1af3b4220f6b admin admin] PATH_INFO: `/v3/projects/f70a0bbf5d2b49e99566672e689657ba` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:16.853963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f5a4de-87d6-4f82-8ecf-1af3b4220f6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f70a0bbf5d2b49e99566672e689657ba)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:16.856203 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f5a4de-87d6-4f82-8ecf-1af3b4220f6b admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:16.930694 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-94f5a4de-87d6-4f82-8ecf-1af3b4220f6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f70a0bbf5d2b49e99566672e689657ba', 'request_id': 'req-94f5a4de-87d6-4f82-8ecf-1af3b4220f6b'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:16.938699 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-94f5a4de-87d6-4f82-8ecf-1af3b4220f6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f70a0bbf5d2b49e99566672e689657ba', 'request_id': 'req-94f5a4de-87d6-4f82-8ecf-1af3b4220f6b'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:16.949233 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-94f5a4de-87d6-4f82-8ecf-1af3b4220f6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f70a0bbf5d2b49e99566672e689657ba', 'request_id': 'req-94f5a4de-87d6-4f82-8ecf-1af3b4220f6b'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:16.962017 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-94f5a4de-87d6-4f82-8ecf-1af3b4220f6b admin admin] The token cache is being invalidate because project f70a0bbf5d2b49e99566672e689657ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:51:16.962425 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-94f5a4de-87d6-4f82-8ecf-1af3b4220f6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:16.963402 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-94f5a4de-87d6-4f82-8ecf-1af3b4220f6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:16.967682 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 826/1648] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:51:16 2022] DELETE /identity/v3/projects/f70a0bbf5d2b49e99566672e689657ba => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:16.970330 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a6fc63-a4f5-44f8-bcf2-b8f01652cd96 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:16.971817 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a6fc63-a4f5-44f8-bcf2-b8f01652cd96 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:16.972106 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a6fc63-a4f5-44f8-bcf2-b8f01652cd96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:17.011197 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-59a6fc63-a4f5-44f8-bcf2-b8f01652cd96 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:17.013570 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-59a6fc63-a4f5-44f8-bcf2-b8f01652cd96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:17.128797 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 823/1649] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:17.137640 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d354b6b3-64f6-46ee-a465-a23aea20c3ec None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:17.137932 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d354b6b3-64f6-46ee-a465-a23aea20c3ec None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:17.138237 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d354b6b3-64f6-46ee-a465-a23aea20c3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:17.153738 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:17.175613 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-d354b6b3-64f6-46ee-a465-a23aea20c3ec None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:17.177229 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-d354b6b3-64f6-46ee-a465-a23aea20c3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:17.197178 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 827/1650] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:17.198449 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a6fc63-a4f5-44f8-bcf2-b8f01652cd96 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:17.214345 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a6fc63-a4f5-44f8-bcf2-b8f01652cd96 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:17.226746 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67708e5-c6a8-4a86-8212-877ef82e2feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:17.227451 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67708e5-c6a8-4a86-8212-877ef82e2feb admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:17.227776 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67708e5-c6a8-4a86-8212-877ef82e2feb admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:17.228198 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67708e5-c6a8-4a86-8212-877ef82e2feb admin admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:17.231324 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67708e5-c6a8-4a86-8212-877ef82e2feb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:51:17.232296 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67708e5-c6a8-4a86-8212-877ef82e2feb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:17.234059 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67708e5-c6a8-4a86-8212-877ef82e2feb admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:17.254584 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 824/1651] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:51:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:51:17.255272 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d354b6b3-64f6-46ee-a465-a23aea20c3ec None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:17.270577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d354b6b3-64f6-46ee-a465-a23aea20c3ec None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:17.292381 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0c8e1a-2cf3-4841-8c92-c353d6619999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:17.293305 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c8e1a-2cf3-4841-8c92-c353d6619999 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:17.293594 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c8e1a-2cf3-4841-8c92-c353d6619999 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:17.293930 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c8e1a-2cf3-4841-8c92-c353d6619999 admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:17.294762 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0c8e1a-2cf3-4841-8c92-c353d6619999 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:17.308338 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0c8e1a-2cf3-4841-8c92-c353d6619999 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:17.408393 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67708e5-c6a8-4a86-8212-877ef82e2feb admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:17.420239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 828/1652] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:51:17 2022] POST /identity/v3/projects => generated 381 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:51:17.421504 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67708e5-c6a8-4a86-8212-877ef82e2feb admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:17.521509 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f75639f-aa84-44c0-9032-a1352e684ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:17.524005 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75639f-aa84-44c0-9032-a1352e684ccf admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:17.524329 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75639f-aa84-44c0-9032-a1352e684ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:17.524611 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75639f-aa84-44c0-9032-a1352e684ccf admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:17.527854 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f75639f-aa84-44c0-9032-a1352e684ccf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:17.530500 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f75639f-aa84-44c0-9032-a1352e684ccf admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:17.570084 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 825/1653] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:51:17 2022] POST /identity/v3/users => generated 291 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:51:17.571435 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0c8e1a-2cf3-4841-8c92-c353d6619999 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:17.591377 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0c8e1a-2cf3-4841-8c92-c353d6619999 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:17.600776 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c4f681-64cc-4a89-9e31-6b8b441a31e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:17.621695 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4f681-64cc-4a89-9e31-6b8b441a31e4 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:17.622182 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4f681-64cc-4a89-9e31-6b8b441a31e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:17.623790 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4f681-64cc-4a89-9e31-6b8b441a31e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:17.624557 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c4f681-64cc-4a89-9e31-6b8b441a31e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:17.626427 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c4f681-64cc-4a89-9e31-6b8b441a31e4 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:17.672047 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 829/1654] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:17 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:51:17.673286 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f75639f-aa84-44c0-9032-a1352e684ccf admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:17.692646 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f75639f-aa84-44c0-9032-a1352e684ccf admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:17.708651 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79042a99-499a-4a03-88b7-92288c217261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:17.709398 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79042a99-499a-4a03-88b7-92288c217261 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:17.709847 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79042a99-499a-4a03-88b7-92288c217261 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:17.710224 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79042a99-499a-4a03-88b7-92288c217261 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:17.712280 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79042a99-499a-4a03-88b7-92288c217261 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:17.715856 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79042a99-499a-4a03-88b7-92288c217261 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:17.754641 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 826/1655] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:17 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:51:17.756173 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c4f681-64cc-4a89-9e31-6b8b441a31e4 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:17.772975 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c4f681-64cc-4a89-9e31-6b8b441a31e4 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:17.787887 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f5b205-2f4e-4cfb-9cc8-4e3bcc410c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:17.789661 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5b205-2f4e-4cfb-9cc8-4e3bcc410c26 admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:17.789941 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5b205-2f4e-4cfb-9cc8-4e3bcc410c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:17.790361 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5b205-2f4e-4cfb-9cc8-4e3bcc410c26 admin admin] PATH_INFO: `/v3/projects/caa2fd38f7d54757a53d1f556767db9b/users/ed7753f56e184b81b1ff1ad895fe3ecf/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:17.818210 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f5b205-2f4e-4cfb-9cc8-4e3bcc410c26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=caa2fd38f7d54757a53d1f556767db9b, user_id=ed7753f56e184b81b1ff1ad895fe3ecf, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:17.823878 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f5b205-2f4e-4cfb-9cc8-4e3bcc410c26 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:17.849731 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 830/1656] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:51:17 2022] PUT /identity/v3/projects/caa2fd38f7d54757a53d1f556767db9b/users/ed7753f56e184b81b1ff1ad895fe3ecf/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:51:17.867133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4648ab-82e2-4057-8d3a-cb8c66b8c02e None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:17.867431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4648ab-82e2-4057-8d3a-cb8c66b8c02e None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:17.867723 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4648ab-82e2-4057-8d3a-cb8c66b8c02e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:17.932059 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-6f4648ab-82e2-4057-8d3a-cb8c66b8c02e None None] MFA Rules not processed for user `ed7753f56e184b81b1ff1ad895fe3ecf`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:17.933353 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-6f4648ab-82e2-4057-8d3a-cb8c66b8c02e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:18.010572 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:18.060587 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 827/1657] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:18.065225 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f5b205-2f4e-4cfb-9cc8-4e3bcc410c26 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:18.156948 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f5b205-2f4e-4cfb-9cc8-4e3bcc410c26 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:18.170588 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8386cf32-3b0b-4b74-834a-8565fc1bb64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:18.171497 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386cf32-3b0b-4b74-834a-8565fc1bb64d admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:18.171964 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386cf32-3b0b-4b74-834a-8565fc1bb64d admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:18.172433 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386cf32-3b0b-4b74-834a-8565fc1bb64d admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:18.173278 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8386cf32-3b0b-4b74-834a-8565fc1bb64d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:18.175300 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8386cf32-3b0b-4b74-834a-8565fc1bb64d admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:18.237934 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 831/1658] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:51:18 2022] POST /identity/v3/projects => generated 381 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:51:18.241875 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4648ab-82e2-4057-8d3a-cb8c66b8c02e None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:18.254683 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4648ab-82e2-4057-8d3a-cb8c66b8c02e None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:18.328074 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa5f615-f0af-45eb-b0b1-0568b1376caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:18.328792 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5f615-f0af-45eb-b0b1-0568b1376caa admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:18.329215 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5f615-f0af-45eb-b0b1-0568b1376caa admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:18.329630 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5f615-f0af-45eb-b0b1-0568b1376caa admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:18.330319 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa5f615-f0af-45eb-b0b1-0568b1376caa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:18.333108 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa5f615-f0af-45eb-b0b1-0568b1376caa admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:18.357896 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 828/1659] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:51:18 2022] POST /identity/v3/users => generated 297 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:51:18.362001 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8386cf32-3b0b-4b74-834a-8565fc1bb64d admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:18.377409 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8386cf32-3b0b-4b74-834a-8565fc1bb64d admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:18.383465 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae5f1dd-e2ef-4262-8e53-e798d2d3b593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:18.384296 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae5f1dd-e2ef-4262-8e53-e798d2d3b593 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:18.384688 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae5f1dd-e2ef-4262-8e53-e798d2d3b593 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:18.385327 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae5f1dd-e2ef-4262-8e53-e798d2d3b593 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:18.386225 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae5f1dd-e2ef-4262-8e53-e798d2d3b593 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:18.388413 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae5f1dd-e2ef-4262-8e53-e798d2d3b593 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:18.403047 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 832/1660] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:18 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:51:18.407037 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa5f615-f0af-45eb-b0b1-0568b1376caa admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:18.418154 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa5f615-f0af-45eb-b0b1-0568b1376caa admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:18.423519 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a119739f-cb17-4727-b5c7-0fccdae4bacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:18.424511 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119739f-cb17-4727-b5c7-0fccdae4bacb admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:18.424830 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119739f-cb17-4727-b5c7-0fccdae4bacb admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:18.425219 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119739f-cb17-4727-b5c7-0fccdae4bacb admin admin] PATH_INFO: `/v3/projects/1ad9918be2a14875ade8dbfca29ef0f6/users/2a457be4fc194fed871e72d1f04dc0ed/roles/0e78444e43c744668178971af4087ca0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:18.449391 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a119739f-cb17-4727-b5c7-0fccdae4bacb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ad9918be2a14875ade8dbfca29ef0f6, user_id=2a457be4fc194fed871e72d1f04dc0ed, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:18.453456 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a119739f-cb17-4727-b5c7-0fccdae4bacb admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:18.484343 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 829/1661] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:51:18 2022] PUT /identity/v3/projects/1ad9918be2a14875ade8dbfca29ef0f6/users/2a457be4fc194fed871e72d1f04dc0ed/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:18.488674 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae5f1dd-e2ef-4262-8e53-e798d2d3b593 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:18.501028 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae5f1dd-e2ef-4262-8e53-e798d2d3b593 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:18.605233 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c6738-e3cd-4972-a5cb-52ce57cdcb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:18.605871 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c6738-e3cd-4972-a5cb-52ce57cdcb03 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:18.606147 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c6738-e3cd-4972-a5cb-52ce57cdcb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:18.606706 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c6738-e3cd-4972-a5cb-52ce57cdcb03 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:18.607548 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813c6738-e3cd-4972-a5cb-52ce57cdcb03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:18.609730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813c6738-e3cd-4972-a5cb-52ce57cdcb03 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:18.631894 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 833/1662] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:18 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 12:51:18.645287 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a119739f-cb17-4727-b5c7-0fccdae4bacb admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:18.659883 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a119739f-cb17-4727-b5c7-0fccdae4bacb admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:18.674821 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e113948-1313-4335-be64-99d690833968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:18.675516 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e113948-1313-4335-be64-99d690833968 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:18.675823 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e113948-1313-4335-be64-99d690833968 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:18.676236 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e113948-1313-4335-be64-99d690833968 admin admin] PATH_INFO: `/v3/projects/1ad9918be2a14875ade8dbfca29ef0f6/users/2a457be4fc194fed871e72d1f04dc0ed/roles/0e78444e43c744668178971af4087ca0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:18.683882 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e113948-1313-4335-be64-99d690833968 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ad9918be2a14875ade8dbfca29ef0f6, user_id=2a457be4fc194fed871e72d1f04dc0ed, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:18.692910 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e113948-1313-4335-be64-99d690833968 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:18.707244 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 830/1663] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:51:18 2022] PUT /identity/v3/projects/1ad9918be2a14875ade8dbfca29ef0f6/users/2a457be4fc194fed871e72d1f04dc0ed/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:18.718321 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dfe96f-2a5e-4c43-81de-19f8eba7bc63 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:18.718615 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dfe96f-2a5e-4c43-81de-19f8eba7bc63 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:18.718893 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dfe96f-2a5e-4c43-81de-19f8eba7bc63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:18.766126 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:18.802823 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-69dfe96f-2a5e-4c43-81de-19f8eba7bc63 None None] MFA Rules not processed for user `2a457be4fc194fed871e72d1f04dc0ed`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:18.804270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-69dfe96f-2a5e-4c43-81de-19f8eba7bc63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:19.035877 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 834/1664] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:18 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:19.059158 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e113948-1313-4335-be64-99d690833968 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:19.062316 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-9e113948-1313-4335-be64-99d690833968 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:19.187924 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e113948-1313-4335-be64-99d690833968 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:19.192255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c30c1b3-7663-46d5-bbef-20e1f1cf8dae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:19.193561 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30c1b3-7663-46d5-bbef-20e1f1cf8dae service cinder] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:19.194058 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30c1b3-7663-46d5-bbef-20e1f1cf8dae service cinder] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:19.195233 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30c1b3-7663-46d5-bbef-20e1f1cf8dae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:19.224857 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c30c1b3-7663-46d5-bbef-20e1f1cf8dae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:19.232056 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c30c1b3-7663-46d5-bbef-20e1f1cf8dae service cinder] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:19.264250 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 831/1665] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:51:19 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:51:19.387615 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dfe96f-2a5e-4c43-81de-19f8eba7bc63 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:19.403278 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dfe96f-2a5e-4c43-81de-19f8eba7bc63 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:19.413028 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf14f0c-8b9a-4aab-9c47-d2f714264d97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:19.414317 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf14f0c-8b9a-4aab-9c47-d2f714264d97 service cinder] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:19.414706 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf14f0c-8b9a-4aab-9c47-d2f714264d97 service cinder] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:19.421212 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf14f0c-8b9a-4aab-9c47-d2f714264d97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:19.542402 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf14f0c-8b9a-4aab-9c47-d2f714264d97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:19.544587 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf14f0c-8b9a-4aab-9c47-d2f714264d97 service cinder] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:19.559496 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 835/1666] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:51:19 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:51:19.911543 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c30c1b3-7663-46d5-bbef-20e1f1cf8dae service cinder] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:19.914801 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-1c30c1b3-7663-46d5-bbef-20e1f1cf8dae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:19.931779 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c30c1b3-7663-46d5-bbef-20e1f1cf8dae service cinder] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:19.937004 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2abca0-657c-44fa-8c4a-6232e75810ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:19.937906 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2abca0-657c-44fa-8c4a-6232e75810ff service cinder] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:19.938379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2abca0-657c-44fa-8c4a-6232e75810ff service cinder] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:19.938841 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2abca0-657c-44fa-8c4a-6232e75810ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:19.941147 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-eb2abca0-657c-44fa-8c4a-6232e75810ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:20.031620 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:20.074970 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2abca0-657c-44fa-8c4a-6232e75810ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:20.077454 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2abca0-657c-44fa-8c4a-6232e75810ff service cinder] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:20.109960 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 832/1667] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:51:19 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:51:23.486605 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f84b8-69ac-4fcb-b855-510068236967 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:23.487470 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f84b8-69ac-4fcb-b855-510068236967 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:23.487731 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f84b8-69ac-4fcb-b855-510068236967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:23.603198 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-cb2f84b8-69ac-4fcb-b855-510068236967 None None] MFA Rules not processed for user `2b5894762d4b4cc99521bc224d2bddd7`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:23.607872 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-cb2f84b8-69ac-4fcb-b855-510068236967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:23.868293 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 836/1668] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:23.870553 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149b222-8209-4d93-b240-43b3a90eddd6 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:23.871374 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149b222-8209-4d93-b240-43b3a90eddd6 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:23.871905 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149b222-8209-4d93-b240-43b3a90eddd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:23.903984 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-7149b222-8209-4d93-b240-43b3a90eddd6 None None] MFA Rules not processed for user `2b5894762d4b4cc99521bc224d2bddd7`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:23.910312 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-7149b222-8209-4d93-b240-43b3a90eddd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:23.940761 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 833/1669] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:23.955692 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f84b8-69ac-4fcb-b855-510068236967 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:23.957223 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-cb2f84b8-69ac-4fcb-b855-510068236967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:24.028101 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:24.130952 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f84b8-69ac-4fcb-b855-510068236967 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:24.153751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce1b1c-b474-4a7a-9a1b-85aca0e15317 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:24.154480 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce1b1c-b474-4a7a-9a1b-85aca0e15317 service nova] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:24.155501 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce1b1c-b474-4a7a-9a1b-85aca0e15317 service nova] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:24.156013 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce1b1c-b474-4a7a-9a1b-85aca0e15317 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:24.172808 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ce1b1c-b474-4a7a-9a1b-85aca0e15317 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:24.193187 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ce1b1c-b474-4a7a-9a1b-85aca0e15317 service nova] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:24.228965 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 837/1670] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:51:23 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:51:25.644290 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4e24d9-c7f3-4063-9deb-798c6a53b03d None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:25.645184 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4e24d9-c7f3-4063-9deb-798c6a53b03d None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:25.645662 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4e24d9-c7f3-4063-9deb-798c6a53b03d None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:25.647296 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 834/1671] 10.222.0.20 () {54 vars in 912 bytes} [Tue Jul 26 12:51:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:51:25.657518 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff975a-f01f-49ff-8e5d-264e1d7fbdca None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:25.657965 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff975a-f01f-49ff-8e5d-264e1d7fbdca None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:25.658583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff975a-f01f-49ff-8e5d-264e1d7fbdca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:25.800703 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-14ff975a-f01f-49ff-8e5d-264e1d7fbdca None None] MFA Rules not processed for user `6830350c6eba4610bbe9ab1dc31f5939`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:25.802144 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-14ff975a-f01f-49ff-8e5d-264e1d7fbdca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:26.068555 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 838/1672] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Jul 26 12:51:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 411 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:26.156753 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4e24d9-c7f3-4063-9deb-798c6a53b03d None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:26.169966 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-8c4e24d9-c7f3-4063-9deb-798c6a53b03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:26.230047 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4e24d9-c7f3-4063-9deb-798c6a53b03d None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:26.268641 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bc6634-2387-48bf-ae62-6783cd633b63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:26.269283 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc6634-2387-48bf-ae62-6783cd633b63 service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:26.274752 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc6634-2387-48bf-ae62-6783cd633b63 service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:26.276631 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc6634-2387-48bf-ae62-6783cd633b63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:26.335553 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bc6634-2387-48bf-ae62-6783cd633b63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:26.348460 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bc6634-2387-48bf-ae62-6783cd633b63 service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:26.394977 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 835/1673] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:51:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:51:27.529968 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f24bf4-a341-412c-b23b-914191802499 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:27.531416 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f24bf4-a341-412c-b23b-914191802499 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:27.532678 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f24bf4-a341-412c-b23b-914191802499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:27.565858 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae862cd-1d88-42b3-8c8c-87ff948fac84 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:27.566367 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae862cd-1d88-42b3-8c8c-87ff948fac84 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:27.566441 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae862cd-1d88-42b3-8c8c-87ff948fac84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:27.645832 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-aae862cd-1d88-42b3-8c8c-87ff948fac84 None None] MFA Rules not processed for user `1401efb776ab424e8ff4f09eac33d92f`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:27.650291 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-aae862cd-1d88-42b3-8c8c-87ff948fac84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:27.705709 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-53f24bf4-a341-412c-b23b-914191802499 None None] MFA Rules not processed for user `17b7e2c1b1b04366b99e9ab892aeb835`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:27.712034 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-53f24bf4-a341-412c-b23b-914191802499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:27.800492 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 836/1674] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:51:27.828719 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae862cd-1d88-42b3-8c8c-87ff948fac84 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:27.830953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-aae862cd-1d88-42b3-8c8c-87ff948fac84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:27.870677 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae862cd-1d88-42b3-8c8c-87ff948fac84 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:27.894468 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2107e2da-5ed8-4230-bab0-b26c2301e599 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:27.895515 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107e2da-5ed8-4230-bab0-b26c2301e599 service nova] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:27.896312 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107e2da-5ed8-4230-bab0-b26c2301e599 service nova] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:27.897122 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107e2da-5ed8-4230-bab0-b26c2301e599 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:27.917157 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 839/1675] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 388 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:27.929237 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2107e2da-5ed8-4230-bab0-b26c2301e599 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:27.930861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2107e2da-5ed8-4230-bab0-b26c2301e599 service nova] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:27.948366 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 837/1676] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:51:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 12:51:28.068396 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f24bf4-a341-412c-b23b-914191802499 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:28.093451 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f24bf4-a341-412c-b23b-914191802499 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:28.112248 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d49f71c-0602-415c-8f76-c10366147d75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:28.113121 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d49f71c-0602-415c-8f76-c10366147d75 service nova] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:28.113363 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d49f71c-0602-415c-8f76-c10366147d75 service nova] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:28.113638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d49f71c-0602-415c-8f76-c10366147d75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:28.141751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d49f71c-0602-415c-8f76-c10366147d75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:28.145797 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d49f71c-0602-415c-8f76-c10366147d75 service nova] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:28.184839 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 840/1677] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:51:28 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:51:28.363624 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5dfbab-cd15-4222-9eac-5245f3f4fd10 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:28.368104 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5dfbab-cd15-4222-9eac-5245f3f4fd10 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:28.368421 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5dfbab-cd15-4222-9eac-5245f3f4fd10 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:28.369915 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 838/1678] 10.222.0.20 () {54 vars in 961 bytes} [Tue Jul 26 12:51:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:51:28.381177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba44ee53-c493-47cc-bddf-acd02ae91ec8 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:28.381640 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba44ee53-c493-47cc-bddf-acd02ae91ec8 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:28.382532 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba44ee53-c493-47cc-bddf-acd02ae91ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:28.421543 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-ba44ee53-c493-47cc-bddf-acd02ae91ec8 None None] MFA Rules not processed for user `ec49a03aed774c439459883bd5442d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:28.426167 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-ba44ee53-c493-47cc-bddf-acd02ae91ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:28.516222 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a18ab5-8462-456a-8fc0-de94f6fd8142 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:28.516423 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a18ab5-8462-456a-8fc0-de94f6fd8142 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:28.516685 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a18ab5-8462-456a-8fc0-de94f6fd8142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:28.566043 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:28.605089 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-a9a18ab5-8462-456a-8fc0-de94f6fd8142 None None] MFA Rules not processed for user `2b5894762d4b4cc99521bc224d2bddd7`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:28.619200 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-a9a18ab5-8462-456a-8fc0-de94f6fd8142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:28.674160 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 839/1679] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:28 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:28.790924 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 841/1680] 10.222.0.20 () {58 vars in 1058 bytes} [Tue Jul 26 12:51:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 411 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:28.806279 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a18ab5-8462-456a-8fc0-de94f6fd8142 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:28.821129 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a18ab5-8462-456a-8fc0-de94f6fd8142 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:28.860607 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3ef987-5a4b-4906-979e-d5f2c31fd66b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec49a03aed774c439459883bd5442d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:28.861930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ef987-5a4b-4906-979e-d5f2c31fd66b service placement] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:28.862234 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ef987-5a4b-4906-979e-d5f2c31fd66b service placement] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:28.863019 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ef987-5a4b-4906-979e-d5f2c31fd66b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:28.873560 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-bf3ef987-5a4b-4906-979e-d5f2c31fd66b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:28.897084 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3ef987-5a4b-4906-979e-d5f2c31fd66b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:28.899389 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3ef987-5a4b-4906-979e-d5f2c31fd66b service placement] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:28.913957 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 840/1681] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:51:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:51:29.317356 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c4242-f1d0-4368-9559-77dbb295858a None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:29.317900 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c4242-f1d0-4368-9559-77dbb295858a None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:29.318553 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c4242-f1d0-4368-9559-77dbb295858a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:29.401320 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-507c4242-f1d0-4368-9559-77dbb295858a None None] MFA Rules not processed for user `7c4ed2f2f6994ae2bb00d5a502715b4f`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:29.402638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-507c4242-f1d0-4368-9559-77dbb295858a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:29.508254 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 842/1682] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:29.520104 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f43648-7d44-4a4a-8f36-a3e87dbdea3a None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:29.520683 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f43648-7d44-4a4a-8f36-a3e87dbdea3a None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:29.521208 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f43648-7d44-4a4a-8f36-a3e87dbdea3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:29.576384 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-c5f43648-7d44-4a4a-8f36-a3e87dbdea3a None None] MFA Rules not processed for user `7c4ed2f2f6994ae2bb00d5a502715b4f`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:29.579124 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-c5f43648-7d44-4a4a-8f36-a3e87dbdea3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:29.599390 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 841/1683] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:29.611751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507c4242-f1d0-4368-9559-77dbb295858a None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:29.645010 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507c4242-f1d0-4368-9559-77dbb295858a None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:29.654214 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f211f3-73f8-494f-ba30-b670a53560db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:29.657377 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f211f3-73f8-494f-ba30-b670a53560db service nova] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:29.657895 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f211f3-73f8-494f-ba30-b670a53560db service nova] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:29.658307 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f211f3-73f8-494f-ba30-b670a53560db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:29.683941 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f211f3-73f8-494f-ba30-b670a53560db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:29.686149 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f211f3-73f8-494f-ba30-b670a53560db service nova] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:29.704217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 843/1684] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:51:29 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:51:30.625647 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85abc3eb-62cd-4fd0-b9ed-b6dce2f8a85d None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:30.626147 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85abc3eb-62cd-4fd0-b9ed-b6dce2f8a85d None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:30.626222 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85abc3eb-62cd-4fd0-b9ed-b6dce2f8a85d None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:30.628142 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 842/1685] 10.222.0.20 () {54 vars in 912 bytes} [Tue Jul 26 12:51:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:51:30.658193 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b4935a-0ce4-45ef-865b-73e700f6d933 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:30.658808 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b4935a-0ce4-45ef-865b-73e700f6d933 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:30.658953 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b4935a-0ce4-45ef-865b-73e700f6d933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:30.732081 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-13b4935a-0ce4-45ef-865b-73e700f6d933 None None] MFA Rules not processed for user `6830350c6eba4610bbe9ab1dc31f5939`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:30.736349 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-13b4935a-0ce4-45ef-865b-73e700f6d933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:30.761744 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 844/1686] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Jul 26 12:51:30 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:30.818046 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85abc3eb-62cd-4fd0-b9ed-b6dce2f8a85d None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:30.821914 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-85abc3eb-62cd-4fd0-b9ed-b6dce2f8a85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:30.865641 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85abc3eb-62cd-4fd0-b9ed-b6dce2f8a85d None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:30.879287 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4660070-eab4-4ddc-b96d-7568964641f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:30.886065 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4660070-eab4-4ddc-b96d-7568964641f5 service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:30.886693 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4660070-eab4-4ddc-b96d-7568964641f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:30.886693 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4660070-eab4-4ddc-b96d-7568964641f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:30.920741 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4660070-eab4-4ddc-b96d-7568964641f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:30.922401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4660070-eab4-4ddc-b96d-7568964641f5 service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:30.955339 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 843/1687] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:51:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:51:32.062802 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bde214-5660-4220-ab3a-62cda8210472 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:32.063826 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bde214-5660-4220-ab3a-62cda8210472 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:32.064399 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bde214-5660-4220-ab3a-62cda8210472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:32.127466 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-f2bde214-5660-4220-ab3a-62cda8210472 None None] MFA Rules not processed for user `1401efb776ab424e8ff4f09eac33d92f`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:32.131436 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-f2bde214-5660-4220-ab3a-62cda8210472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:32.151892 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 845/1688] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:33.259008 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273ea19-0c1a-4d4b-9024-3fa29b3605b1 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:33.259008 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273ea19-0c1a-4d4b-9024-3fa29b3605b1 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:33.259458 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273ea19-0c1a-4d4b-9024-3fa29b3605b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:33.317875 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-c273ea19-0c1a-4d4b-9024-3fa29b3605b1 None None] MFA Rules not processed for user `17b7e2c1b1b04366b99e9ab892aeb835`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:33.320211 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-c273ea19-0c1a-4d4b-9024-3fa29b3605b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:33.340481 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 844/1689] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:35.752013 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b878c9-e5a5-4750-a9a7-0b9c734e1c42 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:35.752973 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b878c9-e5a5-4750-a9a7-0b9c734e1c42 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:35.753591 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b878c9-e5a5-4750-a9a7-0b9c734e1c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:35.921907 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-91b878c9-e5a5-4750-a9a7-0b9c734e1c42 None None] MFA Rules not processed for user `30fc532f464843a3ad35e4d3c387bb3a`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:35.923235 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-91b878c9-e5a5-4750-a9a7-0b9c734e1c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:36.046644 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273ea19-0c1a-4d4b-9024-3fa29b3605b1 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:36.048283 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-c273ea19-0c1a-4d4b-9024-3fa29b3605b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:36.104997 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:36.149195 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:36.177797 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273ea19-0c1a-4d4b-9024-3fa29b3605b1 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:36.178238 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 846/1690] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Jul 26 12:51:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 427 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:36.197124 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d36d37-3ea9-4f70-af42-6c36fe2891e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:36.198217 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d36d37-3ea9-4f70-af42-6c36fe2891e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:36.199241 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d36d37-3ea9-4f70-af42-6c36fe2891e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:36.199841 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d36d37-3ea9-4f70-af42-6c36fe2891e0 admin admin] PATH_INFO: `/v3/users/d9a3980b6df44adb879f3507ca9167da` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:36.203127 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d36d37-3ea9-4f70-af42-6c36fe2891e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9a3980b6df44adb879f3507ca9167da)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:36.208571 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d36d37-3ea9-4f70-af42-6c36fe2891e0 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:36.324612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-a1d36d37-3ea9-4f70-af42-6c36fe2891e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9a3980b6df44adb879f3507ca9167da'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:36.332646 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-a1d36d37-3ea9-4f70-af42-6c36fe2891e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9a3980b6df44adb879f3507ca9167da'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:36.336036 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-a1d36d37-3ea9-4f70-af42-6c36fe2891e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9a3980b6df44adb879f3507ca9167da'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:36.343829 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b878c9-e5a5-4750-a9a7-0b9c734e1c42 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:36.381062 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b878c9-e5a5-4750-a9a7-0b9c734e1c42 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:36.382899 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-a1d36d37-3ea9-4f70-af42-6c36fe2891e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9a3980b6df44adb879f3507ca9167da'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:36.400367 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-a1d36d37-3ea9-4f70-af42-6c36fe2891e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9a3980b6df44adb879f3507ca9167da'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:36.409544 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba7a27d-db54-409a-b6a0-607c34438995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:36.410178 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba7a27d-db54-409a-b6a0-607c34438995 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:36.410485 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba7a27d-db54-409a-b6a0-607c34438995 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:36.410770 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba7a27d-db54-409a-b6a0-607c34438995 admin admin] PATH_INFO: `/v3/users/ed7753f56e184b81b1ff1ad895fe3ecf` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:36.422262 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba7a27d-db54-409a-b6a0-607c34438995 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed7753f56e184b81b1ff1ad895fe3ecf)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:36.422262 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: WARNING py.warnings [None req-fba7a27d-db54-409a-b6a0-607c34438995 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 12:51:36.422262 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: warnings.warn(msg) Jul 26 12:51:36.422262 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: Jul 26 12:51:36.422830 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba7a27d-db54-409a-b6a0-607c34438995 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:36.436022 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 845/1691] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:51:36 2022] DELETE /identity/v3/users/d9a3980b6df44adb879f3507ca9167da => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:36.438367 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad1d49-bad9-4947-bce6-a9817bd28478 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:36.438770 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad1d49-bad9-4947-bce6-a9817bd28478 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:36.439537 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad1d49-bad9-4947-bce6-a9817bd28478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:36.508500 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-39ad1d49-bad9-4947-bce6-a9817bd28478 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:36.509845 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-39ad1d49-bad9-4947-bce6-a9817bd28478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:36.696541 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-fba7a27d-db54-409a-b6a0-607c34438995 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed7753f56e184b81b1ff1ad895fe3ecf'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:36.712759 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-fba7a27d-db54-409a-b6a0-607c34438995 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed7753f56e184b81b1ff1ad895fe3ecf'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:36.713535 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 846/1692] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:36.714669 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad1d49-bad9-4947-bce6-a9817bd28478 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:36.719116 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-fba7a27d-db54-409a-b6a0-607c34438995 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed7753f56e184b81b1ff1ad895fe3ecf'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:36.721551 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-39ad1d49-bad9-4947-bce6-a9817bd28478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:36.785074 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-fba7a27d-db54-409a-b6a0-607c34438995 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed7753f56e184b81b1ff1ad895fe3ecf'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:36.844949 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-fba7a27d-db54-409a-b6a0-607c34438995 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed7753f56e184b81b1ff1ad895fe3ecf'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:36.862370 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 847/1693] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:51:36 2022] DELETE /identity/v3/users/ed7753f56e184b81b1ff1ad895fe3ecf => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:36.862370 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba7a27d-db54-409a-b6a0-607c34438995 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:36.862370 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-fba7a27d-db54-409a-b6a0-607c34438995 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:36.959543 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:37.001717 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba7a27d-db54-409a-b6a0-607c34438995 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:37.011378 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afedba6-d254-482f-a8c6-b307f8480374 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:37.012300 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afedba6-d254-482f-a8c6-b307f8480374 service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:37.012723 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afedba6-d254-482f-a8c6-b307f8480374 service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:37.013354 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afedba6-d254-482f-a8c6-b307f8480374 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:37.015858 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-2afedba6-d254-482f-a8c6-b307f8480374 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:37.031097 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad1d49-bad9-4947-bce6-a9817bd28478 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:37.151172 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afedba6-d254-482f-a8c6-b307f8480374 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:37.154667 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afedba6-d254-482f-a8c6-b307f8480374 service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:37.213771 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 848/1694] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:51:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:51:37.215550 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a265540-9889-45ee-8f85-875b87ffaba4 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:37.216519 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a265540-9889-45ee-8f85-875b87ffaba4 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:37.217375 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a265540-9889-45ee-8f85-875b87ffaba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:37.220160 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e55a477-064f-4b63-bd82-3fc8278a3aeb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd2483f74d7d48b1a7b8ffe32f63abde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:37.221015 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55a477-064f-4b63-bd82-3fc8278a3aeb service swift] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:37.221302 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55a477-064f-4b63-bd82-3fc8278a3aeb service swift] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:37.221579 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55a477-064f-4b63-bd82-3fc8278a3aeb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:37.230985 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-6e55a477-064f-4b63-bd82-3fc8278a3aeb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:37.247192 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-8a265540-9889-45ee-8f85-875b87ffaba4 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:37.261355 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-8a265540-9889-45ee-8f85-875b87ffaba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:37.272287 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 849/1695] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:37.327594 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a265540-9889-45ee-8f85-875b87ffaba4 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:37.329736 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-8a265540-9889-45ee-8f85-875b87ffaba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:37.344102 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e55a477-064f-4b63-bd82-3fc8278a3aeb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:37.345810 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e55a477-064f-4b63-bd82-3fc8278a3aeb service swift] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:37.349175 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a265540-9889-45ee-8f85-875b87ffaba4 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:37.357466 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ef4096-b4bc-412b-b025-8aef5162d75e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:37.358659 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef4096-b4bc-412b-b025-8aef5162d75e service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:37.358659 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef4096-b4bc-412b-b025-8aef5162d75e service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:37.359544 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef4096-b4bc-412b-b025-8aef5162d75e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:37.366740 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 847/1696] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Jul 26 12:51:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 652 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 12:51:37.389066 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:37.392011 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ef4096-b4bc-412b-b025-8aef5162d75e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:37.394965 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ef4096-b4bc-412b-b025-8aef5162d75e service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:37.411412 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 850/1697] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:51:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 12:51:38.216288 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e55a477-064f-4b63-bd82-3fc8278a3aeb service swift] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:38.217904 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-6e55a477-064f-4b63-bd82-3fc8278a3aeb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:38.240513 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e55a477-064f-4b63-bd82-3fc8278a3aeb service swift] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:38.246830 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d100322-89fd-45d2-8a1b-0ede80be8da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:38.247477 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d100322-89fd-45d2-8a1b-0ede80be8da1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:38.248231 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d100322-89fd-45d2-8a1b-0ede80be8da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:38.248668 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d100322-89fd-45d2-8a1b-0ede80be8da1 admin admin] PATH_INFO: `/v3/projects/caa2fd38f7d54757a53d1f556767db9b` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:38.250682 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d100322-89fd-45d2-8a1b-0ede80be8da1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=caa2fd38f7d54757a53d1f556767db9b)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:38.252232 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: WARNING py.warnings [None req-9d100322-89fd-45d2-8a1b-0ede80be8da1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 12:51:38.252232 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: warnings.warn(msg) Jul 26 12:51:38.252232 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: Jul 26 12:51:38.253307 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d100322-89fd-45d2-8a1b-0ede80be8da1 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:38.318700 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-9d100322-89fd-45d2-8a1b-0ede80be8da1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'caa2fd38f7d54757a53d1f556767db9b', 'request_id': 'req-9d100322-89fd-45d2-8a1b-0ede80be8da1'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:38.347245 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ef4096-b4bc-412b-b025-8aef5162d75e service neutron] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:38.347793 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-9d100322-89fd-45d2-8a1b-0ede80be8da1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'caa2fd38f7d54757a53d1f556767db9b', 'request_id': 'req-9d100322-89fd-45d2-8a1b-0ede80be8da1'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:38.349428 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-99ef4096-b4bc-412b-b025-8aef5162d75e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:38.358761 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-9d100322-89fd-45d2-8a1b-0ede80be8da1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'caa2fd38f7d54757a53d1f556767db9b', 'request_id': 'req-9d100322-89fd-45d2-8a1b-0ede80be8da1'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:38.368748 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ef4096-b4bc-412b-b025-8aef5162d75e service neutron] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:38.380020 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-9d100322-89fd-45d2-8a1b-0ede80be8da1 admin admin] The token cache is being invalidate because project caa2fd38f7d54757a53d1f556767db9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:51:38.380307 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-9d100322-89fd-45d2-8a1b-0ede80be8da1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:38.381452 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-9d100322-89fd-45d2-8a1b-0ede80be8da1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:38.384129 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 848/1698] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:51:38 2022] DELETE /identity/v3/projects/caa2fd38f7d54757a53d1f556767db9b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:38.388953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d100322-89fd-45d2-8a1b-0ede80be8da1 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:38.390836 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-9d100322-89fd-45d2-8a1b-0ede80be8da1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:38.422095 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:38.449769 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97b7ff-969b-4144-a42f-bef383824502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:38.450528 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97b7ff-969b-4144-a42f-bef383824502 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:38.450818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97b7ff-969b-4144-a42f-bef383824502 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:38.451317 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97b7ff-969b-4144-a42f-bef383824502 admin admin] PATH_INFO: `/v3/projects/65cb1e6051ad4601870e2d74ce55ce44` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:38.452775 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d97b7ff-969b-4144-a42f-bef383824502 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65cb1e6051ad4601870e2d74ce55ce44)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:38.454806 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d97b7ff-969b-4144-a42f-bef383824502 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:38.494001 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d100322-89fd-45d2-8a1b-0ede80be8da1 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:38.499194 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a4e9ec-2e99-485c-b682-00eada441f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:38.499950 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4e9ec-2e99-485c-b682-00eada441f2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:38.500277 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4e9ec-2e99-485c-b682-00eada441f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:38.500562 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4e9ec-2e99-485c-b682-00eada441f2a admin admin] PATH_INFO: `/v3/users/2a457be4fc194fed871e72d1f04dc0ed` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:38.501820 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a4e9ec-2e99-485c-b682-00eada441f2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a457be4fc194fed871e72d1f04dc0ed)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:38.503848 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a4e9ec-2e99-485c-b682-00eada441f2a admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:38.513180 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-8d97b7ff-969b-4144-a42f-bef383824502 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65cb1e6051ad4601870e2d74ce55ce44', 'request_id': 'req-8d97b7ff-969b-4144-a42f-bef383824502'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:38.521387 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-8d97b7ff-969b-4144-a42f-bef383824502 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65cb1e6051ad4601870e2d74ce55ce44', 'request_id': 'req-8d97b7ff-969b-4144-a42f-bef383824502'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:38.532476 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-8d97b7ff-969b-4144-a42f-bef383824502 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65cb1e6051ad4601870e2d74ce55ce44', 'request_id': 'req-8d97b7ff-969b-4144-a42f-bef383824502'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:38.548541 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-8d97b7ff-969b-4144-a42f-bef383824502 admin admin] The token cache is being invalidate because project 65cb1e6051ad4601870e2d74ce55ce44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:51:38.548923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-8d97b7ff-969b-4144-a42f-bef383824502 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:38.550008 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-8d97b7ff-969b-4144-a42f-bef383824502 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:38.555754 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 851/1699] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:51:38 2022] DELETE /identity/v3/projects/65cb1e6051ad4601870e2d74ce55ce44 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:51:38.558222 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf8d21-ecfc-4da8-9b9a-14015615f652 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:38.558580 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf8d21-ecfc-4da8-9b9a-14015615f652 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:38.558866 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf8d21-ecfc-4da8-9b9a-14015615f652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:38.563924 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-53a4e9ec-2e99-485c-b682-00eada441f2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a457be4fc194fed871e72d1f04dc0ed'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:38.565228 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-53a4e9ec-2e99-485c-b682-00eada441f2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a457be4fc194fed871e72d1f04dc0ed'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:38.566259 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-53a4e9ec-2e99-485c-b682-00eada441f2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a457be4fc194fed871e72d1f04dc0ed'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:38.580964 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-3aaf8d21-ecfc-4da8-9b9a-14015615f652 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:38.582230 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-3aaf8d21-ecfc-4da8-9b9a-14015615f652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:38.584020 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-53a4e9ec-2e99-485c-b682-00eada441f2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a457be4fc194fed871e72d1f04dc0ed'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:38.594220 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-53a4e9ec-2e99-485c-b682-00eada441f2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a457be4fc194fed871e72d1f04dc0ed'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:38.615910 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 849/1700] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:51:38 2022] DELETE /identity/v3/users/2a457be4fc194fed871e72d1f04dc0ed => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:38.670535 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 852/1701] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:38.671922 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ce1c6-8c05-4915-ba02-028cdb861835 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:38.672217 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ce1c6-8c05-4915-ba02-028cdb861835 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:38.672690 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ce1c6-8c05-4915-ba02-028cdb861835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:38.704822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-0c2ce1c6-8c05-4915-ba02-028cdb861835 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:38.706205 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-0c2ce1c6-8c05-4915-ba02-028cdb861835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:38.801440 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 850/1702] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:51:38.807553 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaf8d21-ecfc-4da8-9b9a-14015615f652 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:38.826743 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaf8d21-ecfc-4da8-9b9a-14015615f652 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:38.836558 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8105b-c394-4c89-ab6e-bfb289302a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:38.841730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8105b-c394-4c89-ab6e-bfb289302a36 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:38.842222 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8105b-c394-4c89-ab6e-bfb289302a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:38.842601 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8105b-c394-4c89-ab6e-bfb289302a36 admin admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:38.843478 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b8105b-c394-4c89-ab6e-bfb289302a36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:51:38.852525 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b8105b-c394-4c89-ab6e-bfb289302a36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:38.854181 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b8105b-c394-4c89-ab6e-bfb289302a36 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:38.877746 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 853/1703] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:51:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:51:38.883141 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2ce1c6-8c05-4915-ba02-028cdb861835 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:38.895051 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2ce1c6-8c05-4915-ba02-028cdb861835 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:38.920246 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625ad0ce-0625-45c7-a093-cb38f44deb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:38.921191 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ad0ce-0625-45c7-a093-cb38f44deb07 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:38.921476 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ad0ce-0625-45c7-a093-cb38f44deb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:38.921803 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ad0ce-0625-45c7-a093-cb38f44deb07 admin admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:38.922784 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625ad0ce-0625-45c7-a093-cb38f44deb07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:38.928106 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625ad0ce-0625-45c7-a093-cb38f44deb07 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:39.005334 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 851/1704] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:51:38 2022] POST /identity/v3/projects => generated 401 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:51:39.009855 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8105b-c394-4c89-ab6e-bfb289302a36 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:39.021337 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8105b-c394-4c89-ab6e-bfb289302a36 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:39.042101 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:39.138854 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a6c3d-2db2-4d87-970e-f9338fe2bfe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:39.139502 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a6c3d-2db2-4d87-970e-f9338fe2bfe1 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:39.139837 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a6c3d-2db2-4d87-970e-f9338fe2bfe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:39.140106 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a6c3d-2db2-4d87-970e-f9338fe2bfe1 admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:39.141554 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754a6c3d-2db2-4d87-970e-f9338fe2bfe1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:39.145086 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754a6c3d-2db2-4d87-970e-f9338fe2bfe1 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:39.194898 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625ad0ce-0625-45c7-a093-cb38f44deb07 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:39.196694 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-625ad0ce-0625-45c7-a093-cb38f44deb07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:39.200111 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 854/1705] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:51:39 2022] POST /identity/v3/users => generated 301 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:51:39.201063 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a6c3d-2db2-4d87-970e-f9338fe2bfe1 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:39.234031 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625ad0ce-0625-45c7-a093-cb38f44deb07 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:39.238773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a6c3d-2db2-4d87-970e-f9338fe2bfe1 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:39.254743 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dec239-f52b-4ee7-80ce-87e2c62c5893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:39.255572 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dec239-f52b-4ee7-80ce-87e2c62c5893 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:39.256080 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dec239-f52b-4ee7-80ce-87e2c62c5893 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:39.256634 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dec239-f52b-4ee7-80ce-87e2c62c5893 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:39.258767 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dec239-f52b-4ee7-80ce-87e2c62c5893 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:39.260383 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b4fcd-dcc8-4b00-8403-b7af2d7db32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:39.261039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b4fcd-dcc8-4b00-8403-b7af2d7db32c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:39.261315 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b4fcd-dcc8-4b00-8403-b7af2d7db32c admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:39.261589 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b4fcd-dcc8-4b00-8403-b7af2d7db32c admin admin] PATH_INFO: `/v3/projects/1ad9918be2a14875ade8dbfca29ef0f6` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:39.262062 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dec239-f52b-4ee7-80ce-87e2c62c5893 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:39.265591 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9b4fcd-dcc8-4b00-8403-b7af2d7db32c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ad9918be2a14875ade8dbfca29ef0f6)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:39.275931 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9b4fcd-dcc8-4b00-8403-b7af2d7db32c admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:39.277079 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 855/1706] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:39 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:51:39.282693 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dec239-f52b-4ee7-80ce-87e2c62c5893 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:39.293002 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:39.313775 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dec239-f52b-4ee7-80ce-87e2c62c5893 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:39.319530 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21acd7bb-c6ec-4103-8e09-0c52d00ee840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:39.320530 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21acd7bb-c6ec-4103-8e09-0c52d00ee840 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:39.320998 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21acd7bb-c6ec-4103-8e09-0c52d00ee840 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:39.321502 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21acd7bb-c6ec-4103-8e09-0c52d00ee840 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:39.322277 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21acd7bb-c6ec-4103-8e09-0c52d00ee840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:39.324170 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21acd7bb-c6ec-4103-8e09-0c52d00ee840 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:39.341906 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 856/1707] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:39 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:51:39.342937 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21acd7bb-c6ec-4103-8e09-0c52d00ee840 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:39.376274 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21acd7bb-c6ec-4103-8e09-0c52d00ee840 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:39.409533 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14876fde-3d8e-4f37-b9df-7a1022136f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:39.410236 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14876fde-3d8e-4f37-b9df-7a1022136f6f admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:39.410523 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14876fde-3d8e-4f37-b9df-7a1022136f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:39.410802 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14876fde-3d8e-4f37-b9df-7a1022136f6f admin admin] PATH_INFO: `/v3/projects/8e1107dd67c141d09e37e7e890204e35/users/bb5725dc85f14557a49f8d1c09303692/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:39.466321 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-5d9b4fcd-dcc8-4b00-8403-b7af2d7db32c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ad9918be2a14875ade8dbfca29ef0f6', 'request_id': 'req-5d9b4fcd-dcc8-4b00-8403-b7af2d7db32c'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:39.481169 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-5d9b4fcd-dcc8-4b00-8403-b7af2d7db32c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ad9918be2a14875ade8dbfca29ef0f6', 'request_id': 'req-5d9b4fcd-dcc8-4b00-8403-b7af2d7db32c'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:39.483934 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14876fde-3d8e-4f37-b9df-7a1022136f6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e1107dd67c141d09e37e7e890204e35, user_id=bb5725dc85f14557a49f8d1c09303692, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:39.487161 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14876fde-3d8e-4f37-b9df-7a1022136f6f admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:39.508476 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-5d9b4fcd-dcc8-4b00-8403-b7af2d7db32c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ad9918be2a14875ade8dbfca29ef0f6', 'request_id': 'req-5d9b4fcd-dcc8-4b00-8403-b7af2d7db32c'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:39.525320 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 857/1708] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:51:39 2022] PUT /identity/v3/projects/8e1107dd67c141d09e37e7e890204e35/users/bb5725dc85f14557a49f8d1c09303692/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:39.530676 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c120753-607e-41d3-b3ab-75773e2f798c None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:39.533129 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c120753-607e-41d3-b3ab-75773e2f798c None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:39.534351 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c120753-607e-41d3-b3ab-75773e2f798c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:39.542393 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-5d9b4fcd-dcc8-4b00-8403-b7af2d7db32c admin admin] The token cache is being invalidate because project 1ad9918be2a14875ade8dbfca29ef0f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:51:39.553007 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-5d9b4fcd-dcc8-4b00-8403-b7af2d7db32c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:39.554930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-5d9b4fcd-dcc8-4b00-8403-b7af2d7db32c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:39.556985 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 852/1709] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:51:39 2022] DELETE /identity/v3/projects/1ad9918be2a14875ade8dbfca29ef0f6 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:39.562419 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6a572-6dee-4636-8f1e-1452cce482a6 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:39.562712 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6a572-6dee-4636-8f1e-1452cce482a6 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:39.562991 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6a572-6dee-4636-8f1e-1452cce482a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:39.580587 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-0c120753-607e-41d3-b3ab-75773e2f798c None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:39.582946 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-0c120753-607e-41d3-b3ab-75773e2f798c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:39.588425 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-a9e6a572-6dee-4636-8f1e-1452cce482a6 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:39.589759 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-a9e6a572-6dee-4636-8f1e-1452cce482a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:39.725743 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 858/1710] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:51:39.735510 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c120753-607e-41d3-b3ab-75773e2f798c None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:39.738529 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-0c120753-607e-41d3-b3ab-75773e2f798c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:39.768447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 853/1711] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:51:39.770083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71502dc-60ba-40ba-a1a1-6093e7c4a2da None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:39.770372 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71502dc-60ba-40ba-a1a1-6093e7c4a2da None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:39.770646 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71502dc-60ba-40ba-a1a1-6093e7c4a2da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:39.819547 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-a71502dc-60ba-40ba-a1a1-6093e7c4a2da None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:39.820843 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-a71502dc-60ba-40ba-a1a1-6093e7c4a2da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:39.847143 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 854/1712] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:39.857374 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71502dc-60ba-40ba-a1a1-6093e7c4a2da None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:39.873013 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c120753-607e-41d3-b3ab-75773e2f798c None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:39.878148 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71502dc-60ba-40ba-a1a1-6093e7c4a2da None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:39.881713 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ecf3ba-0f41-4c99-91a0-5cb93d9e835b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:39.882376 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ecf3ba-0f41-4c99-91a0-5cb93d9e835b service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:39.882805 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ecf3ba-0f41-4c99-91a0-5cb93d9e835b service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:39.883085 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ecf3ba-0f41-4c99-91a0-5cb93d9e835b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:39.896468 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705f79e-91e5-41d6-be25-48163c19b315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:39.897680 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705f79e-91e5-41d6-be25-48163c19b315 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:39.897968 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705f79e-91e5-41d6-be25-48163c19b315 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:39.898241 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705f79e-91e5-41d6-be25-48163c19b315 admin admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:39.899074 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1705f79e-91e5-41d6-be25-48163c19b315 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:51:39.899288 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1705f79e-91e5-41d6-be25-48163c19b315 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:39.907984 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1705f79e-91e5-41d6-be25-48163c19b315 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:39.927912 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ecf3ba-0f41-4c99-91a0-5cb93d9e835b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:39.929840 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ecf3ba-0f41-4c99-91a0-5cb93d9e835b service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:39.947659 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:39.949304 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 855/1713] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:51:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:51:39.954431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705f79e-91e5-41d6-be25-48163c19b315 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:39.983420 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 859/1714] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:51:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 12:51:40.000270 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705f79e-91e5-41d6-be25-48163c19b315 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:40.028485 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8a4a5d-8ca8-4258-9305-cc3ff7605f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:40.029521 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a4a5d-8ca8-4258-9305-cc3ff7605f38 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:40.030010 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a4a5d-8ca8-4258-9305-cc3ff7605f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:40.031154 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a4a5d-8ca8-4258-9305-cc3ff7605f38 admin admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:40.032868 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8a4a5d-8ca8-4258-9305-cc3ff7605f38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:40.041334 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8a4a5d-8ca8-4258-9305-cc3ff7605f38 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:40.119667 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 856/1715] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:51:39 2022] POST /identity/v3/projects => generated 365 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:51:40.122080 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ecf3ba-0f41-4c99-91a0-5cb93d9e835b service neutron] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:40.136533 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ecf3ba-0f41-4c99-91a0-5cb93d9e835b service neutron] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:40.356598 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a7dac-a136-4e1e-ad7f-f2a4e188a09e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:40.357794 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a7dac-a136-4e1e-ad7f-f2a4e188a09e admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:40.359223 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a7dac-a136-4e1e-ad7f-f2a4e188a09e admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:40.359223 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a7dac-a136-4e1e-ad7f-f2a4e188a09e admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:40.359392 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837a7dac-a136-4e1e-ad7f-f2a4e188a09e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:40.361939 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837a7dac-a136-4e1e-ad7f-f2a4e188a09e admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:40.401454 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 860/1716] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:51:40 2022] POST /identity/v3/users => generated 283 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:51:40.406202 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8a4a5d-8ca8-4258-9305-cc3ff7605f38 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:40.423782 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8a4a5d-8ca8-4258-9305-cc3ff7605f38 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:40.435979 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dac1eb9-a788-492c-93f8-775a80513bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:40.436741 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac1eb9-a788-492c-93f8-775a80513bb4 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:40.437052 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac1eb9-a788-492c-93f8-775a80513bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:40.437359 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac1eb9-a788-492c-93f8-775a80513bb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:40.438132 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dac1eb9-a788-492c-93f8-775a80513bb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:40.439927 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dac1eb9-a788-492c-93f8-775a80513bb4 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:40.465507 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 857/1717] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:40 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:51:40.469681 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a7dac-a136-4e1e-ad7f-f2a4e188a09e admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:40.485611 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a7dac-a136-4e1e-ad7f-f2a4e188a09e admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:40.493204 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6197ac5e-7db2-4737-b4a5-edea098b800d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:40.493881 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6197ac5e-7db2-4737-b4a5-edea098b800d admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:40.494934 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6197ac5e-7db2-4737-b4a5-edea098b800d admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:40.494934 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6197ac5e-7db2-4737-b4a5-edea098b800d admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:40.495432 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6197ac5e-7db2-4737-b4a5-edea098b800d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:40.497411 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6197ac5e-7db2-4737-b4a5-edea098b800d admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:40.515062 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dac1eb9-a788-492c-93f8-775a80513bb4 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:40.520973 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 861/1718] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:40 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:51:40.528968 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dac1eb9-a788-492c-93f8-775a80513bb4 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:40.535749 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4557ed2-cbf9-4137-b3a9-a0656f67232d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:40.537040 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4557ed2-cbf9-4137-b3a9-a0656f67232d admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:40.537626 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4557ed2-cbf9-4137-b3a9-a0656f67232d admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:40.538250 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4557ed2-cbf9-4137-b3a9-a0656f67232d admin admin] PATH_INFO: `/v3/projects/3d9f9606bc824ec793d7400c391738dc/users/91793690d0414f63ad3c43b29b0032f2/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:40.586728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4557ed2-cbf9-4137-b3a9-a0656f67232d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d9f9606bc824ec793d7400c391738dc, user_id=91793690d0414f63ad3c43b29b0032f2, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:40.596036 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4557ed2-cbf9-4137-b3a9-a0656f67232d admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:40.605655 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c73eb-d7c0-41d8-988d-80e488abbaa6 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:40.606129 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c73eb-d7c0-41d8-988d-80e488abbaa6 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:40.606774 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c73eb-d7c0-41d8-988d-80e488abbaa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:40.643135 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 858/1719] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:51:40 2022] PUT /identity/v3/projects/3d9f9606bc824ec793d7400c391738dc/users/91793690d0414f63ad3c43b29b0032f2/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:40.647021 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479cf5ce-ef3e-42b4-8d14-6bfc861a6a16 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:40.650937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479cf5ce-ef3e-42b4-8d14-6bfc861a6a16 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:40.653256 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479cf5ce-ef3e-42b4-8d14-6bfc861a6a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:40.659395 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-556c73eb-d7c0-41d8-988d-80e488abbaa6 None None] MFA Rules not processed for user `6830350c6eba4610bbe9ab1dc31f5939`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:40.661813 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-556c73eb-d7c0-41d8-988d-80e488abbaa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:40.745771 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-479cf5ce-ef3e-42b4-8d14-6bfc861a6a16 None None] MFA Rules not processed for user `91793690d0414f63ad3c43b29b0032f2`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:40.747367 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-479cf5ce-ef3e-42b4-8d14-6bfc861a6a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:40.761803 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:40.835456 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 862/1720] 10.222.0.46 () {58 vars in 1015 bytes} [Tue Jul 26 12:51:40 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:51:40.999161 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556c73eb-d7c0-41d8-988d-80e488abbaa6 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:41.000250 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 859/1721] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:40 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:41.024044 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479cf5ce-ef3e-42b4-8d14-6bfc861a6a16 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:41.025951 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-479cf5ce-ef3e-42b4-8d14-6bfc861a6a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:41.071365 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479cf5ce-ef3e-42b4-8d14-6bfc861a6a16 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:41.092972 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bef6705-fb3c-4047-a297-63b6ef8fd367 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:41.093670 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef6705-fb3c-4047-a297-63b6ef8fd367 service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:41.093968 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef6705-fb3c-4047-a297-63b6ef8fd367 service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:41.094247 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef6705-fb3c-4047-a297-63b6ef8fd367 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:41.135358 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bef6705-fb3c-4047-a297-63b6ef8fd367 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:41.137030 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bef6705-fb3c-4047-a297-63b6ef8fd367 service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:41.176174 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 860/1722] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:51:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:51:41.176418 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bef6705-fb3c-4047-a297-63b6ef8fd367 service neutron] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:41.210513 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bef6705-fb3c-4047-a297-63b6ef8fd367 service neutron] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:41.245199 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc9366-ce4a-447f-8d3f-6669124094e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:41.250377 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc9366-ce4a-447f-8d3f-6669124094e8 service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:41.253822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc9366-ce4a-447f-8d3f-6669124094e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:41.257641 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc9366-ce4a-447f-8d3f-6669124094e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:41.296741 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abc9366-ce4a-447f-8d3f-6669124094e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:41.298433 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abc9366-ce4a-447f-8d3f-6669124094e8 service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:41.347655 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 861/1723] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:51:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:51:41.349971 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc9366-ce4a-447f-8d3f-6669124094e8 service neutron] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:41.385350 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc9366-ce4a-447f-8d3f-6669124094e8 service neutron] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:41.394664 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556c73eb-d7c0-41d8-988d-80e488abbaa6 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:41.415775 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e966d0-eb56-4455-ba5a-91bb05ec7cb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:41.416709 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e966d0-eb56-4455-ba5a-91bb05ec7cb9 service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:41.417163 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e966d0-eb56-4455-ba5a-91bb05ec7cb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:41.417720 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e966d0-eb56-4455-ba5a-91bb05ec7cb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:41.435064 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79180c94-7eef-49ff-92a0-74a3f25f2ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:41.435747 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79180c94-7eef-49ff-92a0-74a3f25f2ba2 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:41.436039 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79180c94-7eef-49ff-92a0-74a3f25f2ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:41.436321 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79180c94-7eef-49ff-92a0-74a3f25f2ba2 admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:41.437006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79180c94-7eef-49ff-92a0-74a3f25f2ba2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:41.447922 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e966d0-eb56-4455-ba5a-91bb05ec7cb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:41.449035 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79180c94-7eef-49ff-92a0-74a3f25f2ba2 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:41.452717 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e966d0-eb56-4455-ba5a-91bb05ec7cb9 service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:41.490457 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 862/1724] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:51:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:51:41.588385 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 863/1725] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:51:40 2022] POST /identity/v3/projects => generated 363 bytes in 592 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:51:41.589049 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e966d0-eb56-4455-ba5a-91bb05ec7cb9 service neutron] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:41.608365 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e966d0-eb56-4455-ba5a-91bb05ec7cb9 service neutron] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:41.827673 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff5f7b-539d-4e88-9ba7-ce9507217d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:41.830508 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff5f7b-539d-4e88-9ba7-ce9507217d42 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:41.830732 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff5f7b-539d-4e88-9ba7-ce9507217d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:41.831041 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff5f7b-539d-4e88-9ba7-ce9507217d42 admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:41.831860 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff5f7b-539d-4e88-9ba7-ce9507217d42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:41.848240 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff5f7b-539d-4e88-9ba7-ce9507217d42 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:41.937169 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79180c94-7eef-49ff-92a0-74a3f25f2ba2 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:41.948386 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 863/1726] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:51:41 2022] POST /identity/v3/users => generated 288 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:51:41.968733 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79180c94-7eef-49ff-92a0-74a3f25f2ba2 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:41.976193 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb8fa6-9677-4673-a14a-18b2a4118fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:41.977503 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb8fa6-9677-4673-a14a-18b2a4118fff admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:41.977833 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb8fa6-9677-4673-a14a-18b2a4118fff admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:41.978100 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb8fa6-9677-4673-a14a-18b2a4118fff admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:41.979070 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbb8fa6-9677-4673-a14a-18b2a4118fff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:41.981674 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbb8fa6-9677-4673-a14a-18b2a4118fff admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:42.031768 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 864/1727] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:51:42.035879 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff5f7b-539d-4e88-9ba7-ce9507217d42 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:42.064055 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff5f7b-539d-4e88-9ba7-ce9507217d42 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:42.093192 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfab3ca6-400d-41b6-a71a-aa5a0e5018d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:42.095826 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab3ca6-400d-41b6-a71a-aa5a0e5018d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:42.097715 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab3ca6-400d-41b6-a71a-aa5a0e5018d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:42.099041 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab3ca6-400d-41b6-a71a-aa5a0e5018d8 admin admin] PATH_INFO: `/v3/projects/1786cf77109d416589560269c50fc593/users/5910adb4b075418b9fbda31b8af0e33b/roles/0e78444e43c744668178971af4087ca0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:42.156168 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfab3ca6-400d-41b6-a71a-aa5a0e5018d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1786cf77109d416589560269c50fc593, user_id=5910adb4b075418b9fbda31b8af0e33b, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:42.164867 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfab3ca6-400d-41b6-a71a-aa5a0e5018d8 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:42.213257 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 864/1728] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:51:42 2022] PUT /identity/v3/projects/1786cf77109d416589560269c50fc593/users/5910adb4b075418b9fbda31b8af0e33b/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:42.232279 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb8fa6-9677-4673-a14a-18b2a4118fff admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:42.249700 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:42.278027 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb8fa6-9677-4673-a14a-18b2a4118fff admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:42.508419 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563edb51-3f31-49eb-a4a9-0c2ced83b66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:42.509676 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563edb51-3f31-49eb-a4a9-0c2ced83b66e admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:42.510368 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563edb51-3f31-49eb-a4a9-0c2ced83b66e admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:42.510872 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563edb51-3f31-49eb-a4a9-0c2ced83b66e admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:42.512419 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563edb51-3f31-49eb-a4a9-0c2ced83b66e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:42.514505 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563edb51-3f31-49eb-a4a9-0c2ced83b66e admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:42.554843 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 865/1729] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:42 2022] GET /identity/v3/roles => generated 1420 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:51:42.556116 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfab3ca6-400d-41b6-a71a-aa5a0e5018d8 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:42.577902 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfab3ca6-400d-41b6-a71a-aa5a0e5018d8 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:42.608115 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee7fbd-c366-4f5b-b435-1c6fa1aacbd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:42.609361 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee7fbd-c366-4f5b-b435-1c6fa1aacbd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:42.609856 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee7fbd-c366-4f5b-b435-1c6fa1aacbd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:42.610322 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee7fbd-c366-4f5b-b435-1c6fa1aacbd4 admin admin] PATH_INFO: `/v3/projects/1786cf77109d416589560269c50fc593/users/5910adb4b075418b9fbda31b8af0e33b/roles/0e78444e43c744668178971af4087ca0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:42.612893 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ee7fbd-c366-4f5b-b435-1c6fa1aacbd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1786cf77109d416589560269c50fc593, user_id=5910adb4b075418b9fbda31b8af0e33b, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:42.616114 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ee7fbd-c366-4f5b-b435-1c6fa1aacbd4 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:42.655625 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 865/1730] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:51:42 2022] PUT /identity/v3/projects/1786cf77109d416589560269c50fc593/users/5910adb4b075418b9fbda31b8af0e33b/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:42.668181 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2266046-3eee-4373-9d60-c1ff5948c161 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:42.668809 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2266046-3eee-4373-9d60-c1ff5948c161 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:42.669248 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2266046-3eee-4373-9d60-c1ff5948c161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:42.773503 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-c2266046-3eee-4373-9d60-c1ff5948c161 None None] MFA Rules not processed for user `5910adb4b075418b9fbda31b8af0e33b`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:42.778032 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-c2266046-3eee-4373-9d60-c1ff5948c161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:43.069583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 866/1731] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:42 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 402 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:43.084604 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee7fbd-c366-4f5b-b435-1c6fa1aacbd4 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:43.086838 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-28ee7fbd-c366-4f5b-b435-1c6fa1aacbd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:43.166767 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:43.270700 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee7fbd-c366-4f5b-b435-1c6fa1aacbd4 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:43.274378 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8967c0-058e-4c5b-a85b-5ef4544a51d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:43.275049 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8967c0-058e-4c5b-a85b-5ef4544a51d7 service cinder] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:43.275441 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8967c0-058e-4c5b-a85b-5ef4544a51d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:43.275793 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8967c0-058e-4c5b-a85b-5ef4544a51d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:43.288242 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8967c0-058e-4c5b-a85b-5ef4544a51d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:43.289235 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8967c0-058e-4c5b-a85b-5ef4544a51d7 service cinder] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:43.322861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 866/1732] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:51:43 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 12:51:43.463918 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2266046-3eee-4373-9d60-c1ff5948c161 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:43.479925 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2266046-3eee-4373-9d60-c1ff5948c161 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:43.502321 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2080208-c9c3-4a62-86e9-b5de2e27d639 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:43.505062 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2080208-c9c3-4a62-86e9-b5de2e27d639 service cinder] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:43.506436 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2080208-c9c3-4a62-86e9-b5de2e27d639 service cinder] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:43.506963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2080208-c9c3-4a62-86e9-b5de2e27d639 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:43.735321 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2080208-c9c3-4a62-86e9-b5de2e27d639 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:43.738629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2080208-c9c3-4a62-86e9-b5de2e27d639 service cinder] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:43.772874 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 867/1733] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:51:43 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:51:44.298546 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33baf081-242b-4bb8-96f7-3d08357ffb7a None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:44.301952 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33baf081-242b-4bb8-96f7-3d08357ffb7a None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:44.302521 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33baf081-242b-4bb8-96f7-3d08357ffb7a None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:44.304689 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 867/1734] 10.222.0.20 () {54 vars in 921 bytes} [Tue Jul 26 12:51:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:51:44.339380 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e270b8-6fff-4bd1-9e30-6021a886e94c None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:44.339794 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e270b8-6fff-4bd1-9e30-6021a886e94c None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:44.340098 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e270b8-6fff-4bd1-9e30-6021a886e94c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:44.378258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-e5e270b8-6fff-4bd1-9e30-6021a886e94c None None] MFA Rules not processed for user `0f5d6f3c12244c0faf26048513dc49e7`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:44.380409 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-e5e270b8-6fff-4bd1-9e30-6021a886e94c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:44.403065 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:44.529375 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 868/1735] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Jul 26 12:51:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:44.632016 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33baf081-242b-4bb8-96f7-3d08357ffb7a None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:44.634116 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-33baf081-242b-4bb8-96f7-3d08357ffb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:44.656145 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33baf081-242b-4bb8-96f7-3d08357ffb7a None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:44.664189 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e6876-565e-493f-bc37-13912eef1fee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:44.665129 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e6876-565e-493f-bc37-13912eef1fee service nova] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:44.665629 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e6876-565e-493f-bc37-13912eef1fee service nova] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:44.666182 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e6876-565e-493f-bc37-13912eef1fee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:44.677462 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32e6876-565e-493f-bc37-13912eef1fee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:44.679481 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32e6876-565e-493f-bc37-13912eef1fee service nova] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:44.705195 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 868/1736] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:51:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:51:45.606284 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb210d7-78b6-4c71-854e-f97df7798eec None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:45.607284 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb210d7-78b6-4c71-854e-f97df7798eec None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:45.607796 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb210d7-78b6-4c71-854e-f97df7798eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:45.648271 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-4bb210d7-78b6-4c71-854e-f97df7798eec None None] MFA Rules not processed for user `bb5725dc85f14557a49f8d1c09303692`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:45.650617 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-4bb210d7-78b6-4c71-854e-f97df7798eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:45.786400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 869/1737] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:45 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:45.787618 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e6876-565e-493f-bc37-13912eef1fee service nova] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:45.790620 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-c32e6876-565e-493f-bc37-13912eef1fee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:45.873474 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e6876-565e-493f-bc37-13912eef1fee service nova] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:45.881887 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69442c76-8ba5-4c8b-acf2-35c876138ed4 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:45.881887 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69442c76-8ba5-4c8b-acf2-35c876138ed4 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:45.882126 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69442c76-8ba5-4c8b-acf2-35c876138ed4 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:45.883892 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 870/1738] 10.222.0.20 () {54 vars in 921 bytes} [Tue Jul 26 12:51:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 12:51:45.889215 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dbed20-90c8-4987-8b92-f05b0e14aa30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:45.889921 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dbed20-90c8-4987-8b92-f05b0e14aa30 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:45.890468 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dbed20-90c8-4987-8b92-f05b0e14aa30 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:45.890597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dbed20-90c8-4987-8b92-f05b0e14aa30 admin admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:45.893234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35fb149-0a41-40ed-83ff-fe4bdece42ee None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:45.893893 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dbed20-90c8-4987-8b92-f05b0e14aa30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:45.894005 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35fb149-0a41-40ed-83ff-fe4bdece42ee None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:45.894265 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35fb149-0a41-40ed-83ff-fe4bdece42ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:45.897614 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dbed20-90c8-4987-8b92-f05b0e14aa30 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:45.952956 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-f35fb149-0a41-40ed-83ff-fe4bdece42ee None None] MFA Rules not processed for user `0f5d6f3c12244c0faf26048513dc49e7`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:45.953831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-f35fb149-0a41-40ed-83ff-fe4bdece42ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:45.986862 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 871/1739] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Jul 26 12:51:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:45.996028 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 869/1740] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:51:45 2022] POST /identity/v3/projects => generated 403 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:51:46.000467 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35fb149-0a41-40ed-83ff-fe4bdece42ee None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:46.016759 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35fb149-0a41-40ed-83ff-fe4bdece42ee None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:46.019860 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dbed20-90c8-4987-8b92-f05b0e14aa30 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:46.021616 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-82dbed20-90c8-4987-8b92-f05b0e14aa30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:46.217445 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7951107-91cc-4a87-94c2-41966fadea75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:46.218400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7951107-91cc-4a87-94c2-41966fadea75 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:46.218840 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7951107-91cc-4a87-94c2-41966fadea75 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:46.219262 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7951107-91cc-4a87-94c2-41966fadea75 admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:46.220196 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7951107-91cc-4a87-94c2-41966fadea75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:46.222987 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7951107-91cc-4a87-94c2-41966fadea75 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:46.250982 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dbed20-90c8-4987-8b92-f05b0e14aa30 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:46.289773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 872/1741] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:51:45 2022] POST /identity/v3/users => generated 308 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:51:46.294615 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7951107-91cc-4a87-94c2-41966fadea75 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:46.298915 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b268c937-b105-4f5a-9b69-b7f5cf34c717 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:46.300725 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b268c937-b105-4f5a-9b69-b7f5cf34c717 service nova] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:46.301020 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b268c937-b105-4f5a-9b69-b7f5cf34c717 service nova] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:46.313471 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b268c937-b105-4f5a-9b69-b7f5cf34c717 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:46.331460 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7951107-91cc-4a87-94c2-41966fadea75 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:46.339514 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdcb03c-e8c4-4636-98b2-6adcedb968b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:46.340353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcb03c-e8c4-4636-98b2-6adcedb968b9 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:46.348589 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcb03c-e8c4-4636-98b2-6adcedb968b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:46.352262 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcb03c-e8c4-4636-98b2-6adcedb968b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:46.353751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdcb03c-e8c4-4636-98b2-6adcedb968b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:46.355485 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdcb03c-e8c4-4636-98b2-6adcedb968b9 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:46.356616 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b268c937-b105-4f5a-9b69-b7f5cf34c717 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:46.360003 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b268c937-b105-4f5a-9b69-b7f5cf34c717 service nova] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:46.376407 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 873/1742] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:51:46.377421 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdcb03c-e8c4-4636-98b2-6adcedb968b9 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:46.387774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 870/1743] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:51:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 12:51:46.402290 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdcb03c-e8c4-4636-98b2-6adcedb968b9 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:46.411190 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aab23f-58c2-430d-8bef-b3b7b7e41221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:46.411926 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aab23f-58c2-430d-8bef-b3b7b7e41221 admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:46.412264 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aab23f-58c2-430d-8bef-b3b7b7e41221 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:46.413139 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aab23f-58c2-430d-8bef-b3b7b7e41221 admin admin] PATH_INFO: `/v3/projects/061ca49c87b04c3a81334c4cfe1eeb3f/users/6328f4034f7d4703bea27cadcd087a81/roles/0e78444e43c744668178971af4087ca0` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:46.440675 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:46.442628 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aab23f-58c2-430d-8bef-b3b7b7e41221 admin admin] RBAC: Authorizing `identity:create_grant(project_id=061ca49c87b04c3a81334c4cfe1eeb3f, user_id=6328f4034f7d4703bea27cadcd087a81, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:46.450537 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aab23f-58c2-430d-8bef-b3b7b7e41221 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:46.476901 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 874/1744] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:51:46 2022] PUT /identity/v3/projects/061ca49c87b04c3a81334c4cfe1eeb3f/users/6328f4034f7d4703bea27cadcd087a81/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:46.482945 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b268c937-b105-4f5a-9b69-b7f5cf34c717 service nova] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:46.498967 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b268c937-b105-4f5a-9b69-b7f5cf34c717 service nova] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:46.542479 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:46.661607 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc4708-d5d2-41ab-96a6-2746081b2d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:46.662256 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc4708-d5d2-41ab-96a6-2746081b2d81 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:46.662571 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc4708-d5d2-41ab-96a6-2746081b2d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:46.662927 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc4708-d5d2-41ab-96a6-2746081b2d81 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:46.663669 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bc4708-d5d2-41ab-96a6-2746081b2d81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:46.665279 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bc4708-d5d2-41ab-96a6-2746081b2d81 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:46.689543 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 871/1745] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:51:46.694347 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aab23f-58c2-430d-8bef-b3b7b7e41221 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:46.715420 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aab23f-58c2-430d-8bef-b3b7b7e41221 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:46.724443 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2245c312-8df8-460f-b228-cc51119bf9b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:46.725301 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2245c312-8df8-460f-b228-cc51119bf9b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:46.725587 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2245c312-8df8-460f-b228-cc51119bf9b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:46.725865 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2245c312-8df8-460f-b228-cc51119bf9b0 admin admin] PATH_INFO: `/v3/projects/061ca49c87b04c3a81334c4cfe1eeb3f/users/6328f4034f7d4703bea27cadcd087a81/roles/0e78444e43c744668178971af4087ca0` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:46.729120 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2245c312-8df8-460f-b228-cc51119bf9b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=061ca49c87b04c3a81334c4cfe1eeb3f, user_id=6328f4034f7d4703bea27cadcd087a81, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:46.732585 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2245c312-8df8-460f-b228-cc51119bf9b0 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:46.753955 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 875/1746] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:51:46 2022] PUT /identity/v3/projects/061ca49c87b04c3a81334c4cfe1eeb3f/users/6328f4034f7d4703bea27cadcd087a81/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:48.495816 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc4708-d5d2-41ab-96a6-2746081b2d81 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:48.513826 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc4708-d5d2-41ab-96a6-2746081b2d81 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:48.614794 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9671327d-dd6a-4917-b707-129dfe841a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:48.615595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671327d-dd6a-4917-b707-129dfe841a0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:48.616412 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671327d-dd6a-4917-b707-129dfe841a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:48.616764 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671327d-dd6a-4917-b707-129dfe841a0e admin admin] PATH_INFO: `/v3/users/91793690d0414f63ad3c43b29b0032f2` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:48.618323 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9671327d-dd6a-4917-b707-129dfe841a0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=91793690d0414f63ad3c43b29b0032f2)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:48.620887 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9671327d-dd6a-4917-b707-129dfe841a0e admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:48.716807 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-9671327d-dd6a-4917-b707-129dfe841a0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91793690d0414f63ad3c43b29b0032f2'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:48.719108 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-9671327d-dd6a-4917-b707-129dfe841a0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91793690d0414f63ad3c43b29b0032f2'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:48.720825 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-9671327d-dd6a-4917-b707-129dfe841a0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91793690d0414f63ad3c43b29b0032f2'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:48.758279 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-9671327d-dd6a-4917-b707-129dfe841a0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91793690d0414f63ad3c43b29b0032f2'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:48.780872 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-9671327d-dd6a-4917-b707-129dfe841a0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91793690d0414f63ad3c43b29b0032f2'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:48.840974 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 872/1747] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:51:48 2022] DELETE /identity/v3/users/91793690d0414f63ad3c43b29b0032f2 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:48.843329 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd03be-cfd1-443f-9c50-ba8413bb449c None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:48.843855 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd03be-cfd1-443f-9c50-ba8413bb449c None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:48.844058 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd03be-cfd1-443f-9c50-ba8413bb449c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:48.896797 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-6efd03be-cfd1-443f-9c50-ba8413bb449c None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:48.901111 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-6efd03be-cfd1-443f-9c50-ba8413bb449c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:49.129712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 876/1748] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:49.304855 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9671327d-dd6a-4917-b707-129dfe841a0e admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:49.306866 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-9671327d-dd6a-4917-b707-129dfe841a0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:49.431236 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:49.462280 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9671327d-dd6a-4917-b707-129dfe841a0e admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:49.472950 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be42f2da-f077-4b71-a4c8-525f19f13ef4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:49.473945 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42f2da-f077-4b71-a4c8-525f19f13ef4 service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:49.474365 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42f2da-f077-4b71-a4c8-525f19f13ef4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:49.474817 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42f2da-f077-4b71-a4c8-525f19f13ef4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:49.488808 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be42f2da-f077-4b71-a4c8-525f19f13ef4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:49.490518 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be42f2da-f077-4b71-a4c8-525f19f13ef4 service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:49.531075 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 873/1749] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:51:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:51:51.498255 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69fa464-a76d-45d9-8120-537ca5f6a926 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:51.498805 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69fa464-a76d-45d9-8120-537ca5f6a926 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:51.498805 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69fa464-a76d-45d9-8120-537ca5f6a926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:51.542546 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-a69fa464-a76d-45d9-8120-537ca5f6a926 None None] MFA Rules not processed for user `6328f4034f7d4703bea27cadcd087a81`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:51.544389 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-a69fa464-a76d-45d9-8120-537ca5f6a926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:51.651826 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 877/1750] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:51 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:51:51.723674 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be42f2da-f077-4b71-a4c8-525f19f13ef4 service neutron] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:51.725160 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-be42f2da-f077-4b71-a4c8-525f19f13ef4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:51.742867 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be42f2da-f077-4b71-a4c8-525f19f13ef4 service neutron] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:51.750450 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6f9ad-015a-46b9-a40d-ec37e07a950c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:51.751485 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6f9ad-015a-46b9-a40d-ec37e07a950c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:51.752013 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6f9ad-015a-46b9-a40d-ec37e07a950c admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:51.752482 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6f9ad-015a-46b9-a40d-ec37e07a950c admin admin] PATH_INFO: `/v3/projects/3d9f9606bc824ec793d7400c391738dc` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:51.754862 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f6f9ad-015a-46b9-a40d-ec37e07a950c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d9f9606bc824ec793d7400c391738dc)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:51.758910 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f6f9ad-015a-46b9-a40d-ec37e07a950c admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:51.833765 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-f0f6f9ad-015a-46b9-a40d-ec37e07a950c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d9f9606bc824ec793d7400c391738dc', 'request_id': 'req-f0f6f9ad-015a-46b9-a40d-ec37e07a950c'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:51.844579 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-f0f6f9ad-015a-46b9-a40d-ec37e07a950c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d9f9606bc824ec793d7400c391738dc', 'request_id': 'req-f0f6f9ad-015a-46b9-a40d-ec37e07a950c'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:51.855088 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-f0f6f9ad-015a-46b9-a40d-ec37e07a950c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d9f9606bc824ec793d7400c391738dc', 'request_id': 'req-f0f6f9ad-015a-46b9-a40d-ec37e07a950c'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:51.870355 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-f0f6f9ad-015a-46b9-a40d-ec37e07a950c admin admin] The token cache is being invalidate because project 3d9f9606bc824ec793d7400c391738dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:51:51.870650 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-f0f6f9ad-015a-46b9-a40d-ec37e07a950c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:51.871631 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-f0f6f9ad-015a-46b9-a40d-ec37e07a950c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:51.873706 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 874/1751] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:51:51 2022] DELETE /identity/v3/projects/3d9f9606bc824ec793d7400c391738dc => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:51.877929 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69fa464-a76d-45d9-8120-537ca5f6a926 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:51.889258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69fa464-a76d-45d9-8120-537ca5f6a926 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:51.967291 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2bde2b-fe38-4a3c-ad54-f17d57b0f1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:51.968123 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2bde2b-fe38-4a3c-ad54-f17d57b0f1f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:51.968519 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2bde2b-fe38-4a3c-ad54-f17d57b0f1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:51.968889 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2bde2b-fe38-4a3c-ad54-f17d57b0f1f3 admin admin] PATH_INFO: `/v3/users/5910adb4b075418b9fbda31b8af0e33b` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:51.970462 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2bde2b-fe38-4a3c-ad54-f17d57b0f1f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5910adb4b075418b9fbda31b8af0e33b)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:51.972921 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2bde2b-fe38-4a3c-ad54-f17d57b0f1f3 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:51.979310 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:52.081255 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-2b2bde2b-fe38-4a3c-ad54-f17d57b0f1f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5910adb4b075418b9fbda31b8af0e33b'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:52.096111 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-2b2bde2b-fe38-4a3c-ad54-f17d57b0f1f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5910adb4b075418b9fbda31b8af0e33b'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:52.097430 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-2b2bde2b-fe38-4a3c-ad54-f17d57b0f1f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5910adb4b075418b9fbda31b8af0e33b'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:52.129745 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-2b2bde2b-fe38-4a3c-ad54-f17d57b0f1f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5910adb4b075418b9fbda31b8af0e33b'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:52.166614 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-2b2bde2b-fe38-4a3c-ad54-f17d57b0f1f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5910adb4b075418b9fbda31b8af0e33b'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:52.181185 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 878/1752] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:51:51 2022] DELETE /identity/v3/users/5910adb4b075418b9fbda31b8af0e33b => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:52.739949 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f23029a-7c25-411c-b71d-c3ecbe0cbe4f None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:52.740443 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f23029a-7c25-411c-b71d-c3ecbe0cbe4f None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:52.740516 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f23029a-7c25-411c-b71d-c3ecbe0cbe4f None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:52.741874 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 875/1753] 10.222.0.20 () {54 vars in 921 bytes} [Tue Jul 26 12:51:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:51:52.776997 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751618c-b0d2-4c21-89ef-2e930670510b None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:52.778136 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751618c-b0d2-4c21-89ef-2e930670510b None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:52.778542 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751618c-b0d2-4c21-89ef-2e930670510b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:52.802104 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-b751618c-b0d2-4c21-89ef-2e930670510b None None] MFA Rules not processed for user `ec49a03aed774c439459883bd5442d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:52.803737 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-b751618c-b0d2-4c21-89ef-2e930670510b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:52.899915 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 879/1754] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Jul 26 12:51:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:52.920183 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f23029a-7c25-411c-b71d-c3ecbe0cbe4f None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:52.922409 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-7f23029a-7c25-411c-b71d-c3ecbe0cbe4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:52.940937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f23029a-7c25-411c-b71d-c3ecbe0cbe4f None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:52.947073 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8593e-c279-4f65-a880-61b4aab676ad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec49a03aed774c439459883bd5442d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:52.947933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8593e-c279-4f65-a880-61b4aab676ad service placement] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:52.948338 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8593e-c279-4f65-a880-61b4aab676ad service placement] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:52.948717 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8593e-c279-4f65-a880-61b4aab676ad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:52.960006 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb8593e-c279-4f65-a880-61b4aab676ad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:52.962155 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb8593e-c279-4f65-a880-61b4aab676ad service placement] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:52.974795 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 876/1755] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:51:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:51:53.146442 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b751618c-b0d2-4c21-89ef-2e930670510b None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:53.148702 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-b751618c-b0d2-4c21-89ef-2e930670510b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:53.266456 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b751618c-b0d2-4c21-89ef-2e930670510b None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:53.274543 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79c283-6a89-409b-8db0-d2d8b14fc74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:53.275468 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79c283-6a89-409b-8db0-d2d8b14fc74e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:53.275969 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79c283-6a89-409b-8db0-d2d8b14fc74e admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:53.276429 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79c283-6a89-409b-8db0-d2d8b14fc74e admin admin] PATH_INFO: `/v3/projects/1786cf77109d416589560269c50fc593` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:53.278479 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e79c283-6a89-409b-8db0-d2d8b14fc74e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1786cf77109d416589560269c50fc593)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:53.280477 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e79c283-6a89-409b-8db0-d2d8b14fc74e admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:53.309091 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:53.339566 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-8e79c283-6a89-409b-8db0-d2d8b14fc74e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1786cf77109d416589560269c50fc593', 'request_id': 'req-8e79c283-6a89-409b-8db0-d2d8b14fc74e'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:53.346885 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-8e79c283-6a89-409b-8db0-d2d8b14fc74e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1786cf77109d416589560269c50fc593', 'request_id': 'req-8e79c283-6a89-409b-8db0-d2d8b14fc74e'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:53.357237 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-8e79c283-6a89-409b-8db0-d2d8b14fc74e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1786cf77109d416589560269c50fc593', 'request_id': 'req-8e79c283-6a89-409b-8db0-d2d8b14fc74e'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:53.371887 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-8e79c283-6a89-409b-8db0-d2d8b14fc74e admin admin] The token cache is being invalidate because project 1786cf77109d416589560269c50fc593 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:51:53.372279 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-8e79c283-6a89-409b-8db0-d2d8b14fc74e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:53.373264 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-8e79c283-6a89-409b-8db0-d2d8b14fc74e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:53.375169 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 880/1756] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:51:53 2022] DELETE /identity/v3/projects/1786cf77109d416589560269c50fc593 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:53.379975 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6759833e-9db6-4064-88bb-3b7a820c1cfe None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:53.380267 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6759833e-9db6-4064-88bb-3b7a820c1cfe None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:53.380541 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6759833e-9db6-4064-88bb-3b7a820c1cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:53.406163 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-6759833e-9db6-4064-88bb-3b7a820c1cfe None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:53.413469 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-6759833e-9db6-4064-88bb-3b7a820c1cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:53.472555 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 877/1757] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:51:53.481481 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a6e93e-22a7-4454-bc39-1a99a165b228 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:53.481803 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a6e93e-22a7-4454-bc39-1a99a165b228 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:53.482088 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a6e93e-22a7-4454-bc39-1a99a165b228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:53.503050 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-e1a6e93e-22a7-4454-bc39-1a99a165b228 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:53.504237 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-e1a6e93e-22a7-4454-bc39-1a99a165b228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:53.513227 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 881/1758] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:53.517664 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6759833e-9db6-4064-88bb-3b7a820c1cfe None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:53.528518 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6759833e-9db6-4064-88bb-3b7a820c1cfe None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:53.533202 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd5656-3097-493a-9ee7-41a4cc269809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:53.533813 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd5656-3097-493a-9ee7-41a4cc269809 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:53.534093 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd5656-3097-493a-9ee7-41a4cc269809 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:53.534364 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd5656-3097-493a-9ee7-41a4cc269809 admin admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:53.535059 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dd5656-3097-493a-9ee7-41a4cc269809 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:51:53.535353 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dd5656-3097-493a-9ee7-41a4cc269809 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:53.536986 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dd5656-3097-493a-9ee7-41a4cc269809 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:53.549146 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 878/1759] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:51:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:51:53.554036 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a6e93e-22a7-4454-bc39-1a99a165b228 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:53.563300 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a6e93e-22a7-4454-bc39-1a99a165b228 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:53.568348 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0f604-dc79-4644-bcd6-e73dea93d0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:53.569568 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0f604-dc79-4644-bcd6-e73dea93d0b7 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:53.569847 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0f604-dc79-4644-bcd6-e73dea93d0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:53.570114 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0f604-dc79-4644-bcd6-e73dea93d0b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:53.570793 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd0f604-dc79-4644-bcd6-e73dea93d0b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:53.572970 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd0f604-dc79-4644-bcd6-e73dea93d0b7 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:53.577894 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd5656-3097-493a-9ee7-41a4cc269809 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:53.580242 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-45dd5656-3097-493a-9ee7-41a4cc269809 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:53.595285 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd5656-3097-493a-9ee7-41a4cc269809 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:53.601252 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8286ccbb-3714-4ebe-8559-c6e24beece67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:53.601891 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286ccbb-3714-4ebe-8559-c6e24beece67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:53.602172 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286ccbb-3714-4ebe-8559-c6e24beece67 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:53.602444 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286ccbb-3714-4ebe-8559-c6e24beece67 admin admin] PATH_INFO: `/v3/users/bb5725dc85f14557a49f8d1c09303692` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:53.603616 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8286ccbb-3714-4ebe-8559-c6e24beece67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb5725dc85f14557a49f8d1c09303692)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:53.604925 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 882/1760] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:51:53 2022] POST /identity/v3/projects => generated 405 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:51:53.605837 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8286ccbb-3714-4ebe-8559-c6e24beece67 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:53.608764 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0f604-dc79-4644-bcd6-e73dea93d0b7 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:53.642616 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:53.660131 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-8286ccbb-3714-4ebe-8559-c6e24beece67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb5725dc85f14557a49f8d1c09303692'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:53.660854 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-8286ccbb-3714-4ebe-8559-c6e24beece67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb5725dc85f14557a49f8d1c09303692'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:53.661503 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-8286ccbb-3714-4ebe-8559-c6e24beece67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb5725dc85f14557a49f8d1c09303692'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:53.673399 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0f604-dc79-4644-bcd6-e73dea93d0b7 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:53.674467 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-8286ccbb-3714-4ebe-8559-c6e24beece67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb5725dc85f14557a49f8d1c09303692'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:53.679035 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620e47c9-b8b6-4da7-a69e-4be9b461a480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:53.679826 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e47c9-b8b6-4da7-a69e-4be9b461a480 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:53.680155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e47c9-b8b6-4da7-a69e-4be9b461a480 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:53.680447 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e47c9-b8b6-4da7-a69e-4be9b461a480 admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:53.681177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620e47c9-b8b6-4da7-a69e-4be9b461a480 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:53.682001 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-8286ccbb-3714-4ebe-8559-c6e24beece67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb5725dc85f14557a49f8d1c09303692'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:53.683146 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620e47c9-b8b6-4da7-a69e-4be9b461a480 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:53.694782 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 879/1761] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:51:53 2022] DELETE /identity/v3/users/bb5725dc85f14557a49f8d1c09303692 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:53.714152 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 883/1762] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:51:53 2022] POST /identity/v3/users => generated 303 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:51:53.715510 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8286ccbb-3714-4ebe-8559-c6e24beece67 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:53.717005 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-8286ccbb-3714-4ebe-8559-c6e24beece67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:53.788348 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8286ccbb-3714-4ebe-8559-c6e24beece67 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:53.793737 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a9412d-1bfd-4bff-9772-07f4f95b5f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:53.794401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a9412d-1bfd-4bff-9772-07f4f95b5f24 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:53.794683 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a9412d-1bfd-4bff-9772-07f4f95b5f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:53.794955 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a9412d-1bfd-4bff-9772-07f4f95b5f24 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:53.795681 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a9412d-1bfd-4bff-9772-07f4f95b5f24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:53.797300 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a9412d-1bfd-4bff-9772-07f4f95b5f24 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:53.810741 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 880/1763] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 12:51:53.811952 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620e47c9-b8b6-4da7-a69e-4be9b461a480 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:53.823440 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620e47c9-b8b6-4da7-a69e-4be9b461a480 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:53.829054 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ea0c71-fedf-4214-af4e-badbe5d16799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:53.829676 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea0c71-fedf-4214-af4e-badbe5d16799 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:53.829973 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea0c71-fedf-4214-af4e-badbe5d16799 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:53.830233 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea0c71-fedf-4214-af4e-badbe5d16799 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:53.830939 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ea0c71-fedf-4214-af4e-badbe5d16799 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:53.832576 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ea0c71-fedf-4214-af4e-badbe5d16799 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:53.842854 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 884/1764] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:51:53.847138 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a9412d-1bfd-4bff-9772-07f4f95b5f24 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:53.858433 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a9412d-1bfd-4bff-9772-07f4f95b5f24 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:53.863932 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6319bbf9-f72d-4c8c-b0b7-8f52a45160c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:53.864629 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319bbf9-f72d-4c8c-b0b7-8f52a45160c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:53.864979 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319bbf9-f72d-4c8c-b0b7-8f52a45160c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:53.865266 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319bbf9-f72d-4c8c-b0b7-8f52a45160c5 admin admin] PATH_INFO: `/v3/projects/c95ed175d48e45ec9755854bf44948d5/users/10678e4317e740a7ba3994e2fd993515/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:53.878809 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ea0c71-fedf-4214-af4e-badbe5d16799 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:53.880957 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-94ea0c71-fedf-4214-af4e-badbe5d16799 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:53.897871 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6319bbf9-f72d-4c8c-b0b7-8f52a45160c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c95ed175d48e45ec9755854bf44948d5, user_id=10678e4317e740a7ba3994e2fd993515, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:53.904139 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6319bbf9-f72d-4c8c-b0b7-8f52a45160c5 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:53.913593 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ea0c71-fedf-4214-af4e-badbe5d16799 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:53.928825 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 881/1765] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:51:53 2022] PUT /identity/v3/projects/c95ed175d48e45ec9755854bf44948d5/users/10678e4317e740a7ba3994e2fd993515/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:53.930553 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a5dec-f22a-4a33-99bf-26119b166af4 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:53.930992 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a5dec-f22a-4a33-99bf-26119b166af4 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:53.931256 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a5dec-f22a-4a33-99bf-26119b166af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:53.970257 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-d38a5dec-f22a-4a33-99bf-26119b166af4 None None] MFA Rules not processed for user `10678e4317e740a7ba3994e2fd993515`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:53.971754 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-d38a5dec-f22a-4a33-99bf-26119b166af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:54.006823 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a612e-309d-4ca5-8091-c43881a1337b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:54.007921 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a612e-309d-4ca5-8091-c43881a1337b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:54.008369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a612e-309d-4ca5-8091-c43881a1337b admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:54.009332 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a612e-309d-4ca5-8091-c43881a1337b admin admin] PATH_INFO: `/v3/projects/8e1107dd67c141d09e37e7e890204e35` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:54.011363 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270a612e-309d-4ca5-8091-c43881a1337b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e1107dd67c141d09e37e7e890204e35)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:54.024982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270a612e-309d-4ca5-8091-c43881a1337b admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:54.028609 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:54.077890 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 882/1766] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:53 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:54.078947 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a5dec-f22a-4a33-99bf-26119b166af4 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:54.097700 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a5dec-f22a-4a33-99bf-26119b166af4 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:54.118754 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:54.121049 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-270a612e-309d-4ca5-8091-c43881a1337b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e1107dd67c141d09e37e7e890204e35', 'request_id': 'req-270a612e-309d-4ca5-8091-c43881a1337b'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:54.138213 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-270a612e-309d-4ca5-8091-c43881a1337b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e1107dd67c141d09e37e7e890204e35', 'request_id': 'req-270a612e-309d-4ca5-8091-c43881a1337b'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:54.149096 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-270a612e-309d-4ca5-8091-c43881a1337b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e1107dd67c141d09e37e7e890204e35', 'request_id': 'req-270a612e-309d-4ca5-8091-c43881a1337b'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:54.171917 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-270a612e-309d-4ca5-8091-c43881a1337b admin admin] The token cache is being invalidate because project 8e1107dd67c141d09e37e7e890204e35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:51:54.172244 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-270a612e-309d-4ca5-8091-c43881a1337b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:54.173030 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-270a612e-309d-4ca5-8091-c43881a1337b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:54.175001 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 885/1767] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:51:53 2022] DELETE /identity/v3/projects/8e1107dd67c141d09e37e7e890204e35 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:54.184324 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a612e-309d-4ca5-8091-c43881a1337b admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:54.184324 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-270a612e-309d-4ca5-8091-c43881a1337b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:54.202511 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ac221-9be8-435e-8b37-3916e13cf157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:54.203271 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ac221-9be8-435e-8b37-3916e13cf157 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:54.203998 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ac221-9be8-435e-8b37-3916e13cf157 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:54.203998 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ac221-9be8-435e-8b37-3916e13cf157 admin admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:54.204559 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8ac221-9be8-435e-8b37-3916e13cf157 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:54.206962 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8ac221-9be8-435e-8b37-3916e13cf157 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:54.257076 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 883/1768] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:51:54 2022] POST /identity/v3/projects => generated 407 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:51:54.261865 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ac221-9be8-435e-8b37-3916e13cf157 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:54.264332 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-4c8ac221-9be8-435e-8b37-3916e13cf157 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:54.312538 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a612e-309d-4ca5-8091-c43881a1337b admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:54.355055 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ac221-9be8-435e-8b37-3916e13cf157 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:54.362389 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af4675b-ce91-4e5f-81f5-508c468541dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:54.363033 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af4675b-ce91-4e5f-81f5-508c468541dd admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:54.363337 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af4675b-ce91-4e5f-81f5-508c468541dd admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:54.363600 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af4675b-ce91-4e5f-81f5-508c468541dd admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:54.364334 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af4675b-ce91-4e5f-81f5-508c468541dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:54.366379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af4675b-ce91-4e5f-81f5-508c468541dd admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:54.403772 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 884/1769] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:51:54 2022] POST /identity/v3/users => generated 310 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:51:54.407746 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af4675b-ce91-4e5f-81f5-508c468541dd admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:54.414535 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28258101-3b11-4243-bbef-730d3393985e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:54.415294 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28258101-3b11-4243-bbef-730d3393985e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:54.415667 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28258101-3b11-4243-bbef-730d3393985e admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:54.416127 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28258101-3b11-4243-bbef-730d3393985e admin admin] PATH_INFO: `/v3/users/6328f4034f7d4703bea27cadcd087a81` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:54.417396 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28258101-3b11-4243-bbef-730d3393985e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6328f4034f7d4703bea27cadcd087a81)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:54.419907 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28258101-3b11-4243-bbef-730d3393985e admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:54.430354 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af4675b-ce91-4e5f-81f5-508c468541dd admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:54.442752 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab21847-cba2-4eea-be57-2aaf90394678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:54.444278 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab21847-cba2-4eea-be57-2aaf90394678 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:54.444551 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab21847-cba2-4eea-be57-2aaf90394678 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:54.444824 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab21847-cba2-4eea-be57-2aaf90394678 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:54.445541 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab21847-cba2-4eea-be57-2aaf90394678 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:54.447577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab21847-cba2-4eea-be57-2aaf90394678 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:54.460566 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 885/1770] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 12:51:54.461409 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab21847-cba2-4eea-be57-2aaf90394678 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:54.463248 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-3ab21847-cba2-4eea-be57-2aaf90394678 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:54.476931 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab21847-cba2-4eea-be57-2aaf90394678 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:54.481748 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6692c3b4-a9ac-47d0-9b50-475b7856a076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:54.482389 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692c3b4-a9ac-47d0-9b50-475b7856a076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:54.482665 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692c3b4-a9ac-47d0-9b50-475b7856a076 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:54.482936 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692c3b4-a9ac-47d0-9b50-475b7856a076 admin admin] PATH_INFO: `/v3/users/9a0faf8c26394a5b9d5c7e23756a19ea` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:54.484140 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6692c3b4-a9ac-47d0-9b50-475b7856a076 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a0faf8c26394a5b9d5c7e23756a19ea)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:54.486019 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:54.486156 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6692c3b4-a9ac-47d0-9b50-475b7856a076 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:54.496749 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-28258101-3b11-4243-bbef-730d3393985e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6328f4034f7d4703bea27cadcd087a81'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:54.508068 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-28258101-3b11-4243-bbef-730d3393985e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6328f4034f7d4703bea27cadcd087a81'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:54.509650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-28258101-3b11-4243-bbef-730d3393985e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6328f4034f7d4703bea27cadcd087a81'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:54.519091 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-28258101-3b11-4243-bbef-730d3393985e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6328f4034f7d4703bea27cadcd087a81'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:54.532953 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-28258101-3b11-4243-bbef-730d3393985e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6328f4034f7d4703bea27cadcd087a81'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:54.538802 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-6692c3b4-a9ac-47d0-9b50-475b7856a076 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a0faf8c26394a5b9d5c7e23756a19ea'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:54.539588 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-6692c3b4-a9ac-47d0-9b50-475b7856a076 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a0faf8c26394a5b9d5c7e23756a19ea'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:54.540241 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-6692c3b4-a9ac-47d0-9b50-475b7856a076 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a0faf8c26394a5b9d5c7e23756a19ea'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:54.540397 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 886/1771] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:51:54 2022] DELETE /identity/v3/users/6328f4034f7d4703bea27cadcd087a81 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:51:54.541368 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28258101-3b11-4243-bbef-730d3393985e admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:54.543150 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-28258101-3b11-4243-bbef-730d3393985e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:54.561497 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-6692c3b4-a9ac-47d0-9b50-475b7856a076 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a0faf8c26394a5b9d5c7e23756a19ea'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:54.563908 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:54.571016 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-6692c3b4-a9ac-47d0-9b50-475b7856a076 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a0faf8c26394a5b9d5c7e23756a19ea'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:54.589454 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 886/1772] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:51:54 2022] DELETE /identity/v3/users/9a0faf8c26394a5b9d5c7e23756a19ea => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:54.593947 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a151c35-f963-4f76-9d50-81b3b75a2290 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:54.594235 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a151c35-f963-4f76-9d50-81b3b75a2290 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:54.594511 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a151c35-f963-4f76-9d50-81b3b75a2290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:54.626552 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-1a151c35-f963-4f76-9d50-81b3b75a2290 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:54.627909 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-1a151c35-f963-4f76-9d50-81b3b75a2290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:54.628867 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28258101-3b11-4243-bbef-730d3393985e admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:54.696814 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c9726e-b96a-405a-ab32-575f25d8af7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:54.697653 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9726e-b96a-405a-ab32-575f25d8af7c admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:54.698042 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9726e-b96a-405a-ab32-575f25d8af7c admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:54.698425 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9726e-b96a-405a-ab32-575f25d8af7c admin admin] PATH_INFO: `/v3/projects/6a8e0f952ee540fdb963c8b5c891042b/users/4d3b5381f14f4f1d93b4fc0cfefc6b47/roles/0e78444e43c744668178971af4087ca0` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:54.714981 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 887/1773] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:54.725550 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a151c35-f963-4f76-9d50-81b3b75a2290 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:54.726633 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c9726e-b96a-405a-ab32-575f25d8af7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a8e0f952ee540fdb963c8b5c891042b, user_id=4d3b5381f14f4f1d93b4fc0cfefc6b47, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:54.726793 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-1a151c35-f963-4f76-9d50-81b3b75a2290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:54.729448 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c9726e-b96a-405a-ab32-575f25d8af7c admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:54.750834 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 887/1774] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:51:54 2022] PUT /identity/v3/projects/6a8e0f952ee540fdb963c8b5c891042b/users/4d3b5381f14f4f1d93b4fc0cfefc6b47/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:54.752109 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c9726e-b96a-405a-ab32-575f25d8af7c admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:54.754443 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-38c9726e-b96a-405a-ab32-575f25d8af7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:54.795550 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a151c35-f963-4f76-9d50-81b3b75a2290 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:54.825157 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:54.844113 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c9726e-b96a-405a-ab32-575f25d8af7c admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:54.848899 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af52f21a-fcc2-44cf-8f02-4d597dc254e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:54.849621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af52f21a-fcc2-44cf-8f02-4d597dc254e6 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:54.850065 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af52f21a-fcc2-44cf-8f02-4d597dc254e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:54.850405 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af52f21a-fcc2-44cf-8f02-4d597dc254e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:54.851177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af52f21a-fcc2-44cf-8f02-4d597dc254e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:54.852908 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af52f21a-fcc2-44cf-8f02-4d597dc254e6 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:54.854515 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a591f3ce-53a3-4824-9c91-4e7aeb556145 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:54.855216 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a591f3ce-53a3-4824-9c91-4e7aeb556145 service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:54.855521 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a591f3ce-53a3-4824-9c91-4e7aeb556145 service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:54.855884 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a591f3ce-53a3-4824-9c91-4e7aeb556145 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:54.864041 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 888/1775] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:51:54.864948 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af52f21a-fcc2-44cf-8f02-4d597dc254e6 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:54.866489 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a591f3ce-53a3-4824-9c91-4e7aeb556145 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:54.868314 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a591f3ce-53a3-4824-9c91-4e7aeb556145 service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:54.869256 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-af52f21a-fcc2-44cf-8f02-4d597dc254e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:54.879286 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 888/1776] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:51:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:51:54.880446 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a591f3ce-53a3-4824-9c91-4e7aeb556145 service neutron] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:54.882431 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af52f21a-fcc2-44cf-8f02-4d597dc254e6 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:54.887222 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e56994-7e55-41c6-a5e2-ee9eaf54f855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:54.887901 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e56994-7e55-41c6-a5e2-ee9eaf54f855 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:54.888526 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e56994-7e55-41c6-a5e2-ee9eaf54f855 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:54.888526 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e56994-7e55-41c6-a5e2-ee9eaf54f855 admin admin] PATH_INFO: `/v3/projects/061ca49c87b04c3a81334c4cfe1eeb3f` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:54.889614 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e56994-7e55-41c6-a5e2-ee9eaf54f855 admin admin] RBAC: Authorizing `identity:delete_project(project_id=061ca49c87b04c3a81334c4cfe1eeb3f)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:54.891291 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a591f3ce-53a3-4824-9c91-4e7aeb556145 service neutron] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:54.896061 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e56994-7e55-41c6-a5e2-ee9eaf54f855 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:54.900480 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb888a9-f14a-4242-8613-0fd489117321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:54.901649 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb888a9-f14a-4242-8613-0fd489117321 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:54.901942 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb888a9-f14a-4242-8613-0fd489117321 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:54.902267 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb888a9-f14a-4242-8613-0fd489117321 admin admin] PATH_INFO: `/v3/projects/6a8e0f952ee540fdb963c8b5c891042b/users/4d3b5381f14f4f1d93b4fc0cfefc6b47/roles/0e78444e43c744668178971af4087ca0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:54.904377 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb888a9-f14a-4242-8613-0fd489117321 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a8e0f952ee540fdb963c8b5c891042b, user_id=4d3b5381f14f4f1d93b4fc0cfefc6b47, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:54.907199 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb888a9-f14a-4242-8613-0fd489117321 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:54.919715 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 889/1777] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:51:54 2022] PUT /identity/v3/projects/6a8e0f952ee540fdb963c8b5c891042b/users/4d3b5381f14f4f1d93b4fc0cfefc6b47/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:54.931530 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837d0a1-eef4-457d-9d04-3a60c18a4028 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:54.931739 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837d0a1-eef4-457d-9d04-3a60c18a4028 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:54.931997 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837d0a1-eef4-457d-9d04-3a60c18a4028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:54.953017 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-98e56994-7e55-41c6-a5e2-ee9eaf54f855 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '061ca49c87b04c3a81334c4cfe1eeb3f', 'request_id': 'req-98e56994-7e55-41c6-a5e2-ee9eaf54f855'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:54.961830 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-98e56994-7e55-41c6-a5e2-ee9eaf54f855 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '061ca49c87b04c3a81334c4cfe1eeb3f', 'request_id': 'req-98e56994-7e55-41c6-a5e2-ee9eaf54f855'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:54.967496 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-6837d0a1-eef4-457d-9d04-3a60c18a4028 None None] MFA Rules not processed for user `4d3b5381f14f4f1d93b4fc0cfefc6b47`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:54.968946 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-6837d0a1-eef4-457d-9d04-3a60c18a4028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:54.971271 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-98e56994-7e55-41c6-a5e2-ee9eaf54f855 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '061ca49c87b04c3a81334c4cfe1eeb3f', 'request_id': 'req-98e56994-7e55-41c6-a5e2-ee9eaf54f855'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:54.986220 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-98e56994-7e55-41c6-a5e2-ee9eaf54f855 admin admin] The token cache is being invalidate because project 061ca49c87b04c3a81334c4cfe1eeb3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:51:54.986816 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-98e56994-7e55-41c6-a5e2-ee9eaf54f855 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:54.987677 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-98e56994-7e55-41c6-a5e2-ee9eaf54f855 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:54.989495 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 889/1778] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:51:54 2022] DELETE /identity/v3/projects/061ca49c87b04c3a81334c4cfe1eeb3f => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:54.991417 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:54.995713 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b3e3ea-165e-476d-97db-328210a0adb3 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:54.996008 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b3e3ea-165e-476d-97db-328210a0adb3 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:54.996698 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b3e3ea-165e-476d-97db-328210a0adb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:55.024235 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-48b3e3ea-165e-476d-97db-328210a0adb3 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:55.025467 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-48b3e3ea-165e-476d-97db-328210a0adb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:55.061621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 890/1779] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:54 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:51:55.073376 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6837d0a1-eef4-457d-9d04-3a60c18a4028 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:55.078745 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-6837d0a1-eef4-457d-9d04-3a60c18a4028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:55.114906 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 890/1780] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:51:55.122537 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900b758-27fa-4ebf-a8a2-8411b78451b6 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:55.122833 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900b758-27fa-4ebf-a8a2-8411b78451b6 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:55.123367 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900b758-27fa-4ebf-a8a2-8411b78451b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:55.146204 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-d900b758-27fa-4ebf-a8a2-8411b78451b6 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:55.147523 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-d900b758-27fa-4ebf-a8a2-8411b78451b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:55.157346 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 891/1781] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:55.169763 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d900b758-27fa-4ebf-a8a2-8411b78451b6 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:55.170052 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6837d0a1-eef4-457d-9d04-3a60c18a4028 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:55.174376 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8178237f-6c90-4455-98ea-c9b72382add2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:55.175201 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8178237f-6c90-4455-98ea-c9b72382add2 service cinder] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:55.175486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8178237f-6c90-4455-98ea-c9b72382add2 service cinder] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:55.175813 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8178237f-6c90-4455-98ea-c9b72382add2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:55.182420 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d900b758-27fa-4ebf-a8a2-8411b78451b6 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:55.191904 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8178237f-6c90-4455-98ea-c9b72382add2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:55.192108 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5de9ff-6c17-4c82-be15-766a64e2e7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:55.192670 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5de9ff-6c17-4c82-be15-766a64e2e7f7 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:55.192943 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5de9ff-6c17-4c82-be15-766a64e2e7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:55.193217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5de9ff-6c17-4c82-be15-766a64e2e7f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:55.193406 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8178237f-6c90-4455-98ea-c9b72382add2 service cinder] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:55.194459 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5de9ff-6c17-4c82-be15-766a64e2e7f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:51:55.194763 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5de9ff-6c17-4c82-be15-766a64e2e7f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:55.196560 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5de9ff-6c17-4c82-be15-766a64e2e7f7 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:55.205709 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 891/1782] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:51:55 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:51:55.211592 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 892/1783] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:51:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:51:55.216667 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5de9ff-6c17-4c82-be15-766a64e2e7f7 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:55.218131 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8178237f-6c90-4455-98ea-c9b72382add2 service cinder] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:55.219662 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-8178237f-6c90-4455-98ea-c9b72382add2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:55.228708 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5de9ff-6c17-4c82-be15-766a64e2e7f7 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:55.234292 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f256f-f454-4ae1-a9da-94932d48984d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:55.235151 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f256f-f454-4ae1-a9da-94932d48984d admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:55.236093 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f256f-f454-4ae1-a9da-94932d48984d admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:55.236582 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f256f-f454-4ae1-a9da-94932d48984d admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:55.237401 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1f256f-f454-4ae1-a9da-94932d48984d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:55.239571 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1f256f-f454-4ae1-a9da-94932d48984d admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:55.244102 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8178237f-6c90-4455-98ea-c9b72382add2 service cinder] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:55.267502 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c4885f-4986-4367-89cb-5c14ff8e2c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:55.269294 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4885f-4986-4367-89cb-5c14ff8e2c00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:55.269911 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4885f-4986-4367-89cb-5c14ff8e2c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:55.270884 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4885f-4986-4367-89cb-5c14ff8e2c00 admin admin] PATH_INFO: `/v3/projects/2978eaf529f540eab10fde695e9f049d` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:55.272591 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c4885f-4986-4367-89cb-5c14ff8e2c00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2978eaf529f540eab10fde695e9f049d)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:55.274784 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c4885f-4986-4367-89cb-5c14ff8e2c00 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:55.291471 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 893/1784] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:51:55 2022] POST /identity/v3/projects => generated 365 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:51:55.296310 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f256f-f454-4ae1-a9da-94932d48984d admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:55.343860 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:55.347042 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-38c4885f-4986-4367-89cb-5c14ff8e2c00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2978eaf529f540eab10fde695e9f049d', 'request_id': 'req-38c4885f-4986-4367-89cb-5c14ff8e2c00'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:55.360055 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-38c4885f-4986-4367-89cb-5c14ff8e2c00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2978eaf529f540eab10fde695e9f049d', 'request_id': 'req-38c4885f-4986-4367-89cb-5c14ff8e2c00'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:55.380024 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-38c4885f-4986-4367-89cb-5c14ff8e2c00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2978eaf529f540eab10fde695e9f049d', 'request_id': 'req-38c4885f-4986-4367-89cb-5c14ff8e2c00'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:55.404687 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-38c4885f-4986-4367-89cb-5c14ff8e2c00 admin admin] The token cache is being invalidate because project 2978eaf529f540eab10fde695e9f049d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:51:55.404870 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-38c4885f-4986-4367-89cb-5c14ff8e2c00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:55.405524 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-38c4885f-4986-4367-89cb-5c14ff8e2c00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:55.407290 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f256f-f454-4ae1-a9da-94932d48984d admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:55.410853 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 892/1785] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:51:55 2022] DELETE /identity/v3/projects/2978eaf529f540eab10fde695e9f049d => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:55.411723 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c4885f-4986-4367-89cb-5c14ff8e2c00 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:55.414095 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-38c4885f-4986-4367-89cb-5c14ff8e2c00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:55.485998 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:55.508170 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c4885f-4986-4367-89cb-5c14ff8e2c00 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:55.509655 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0bcea2-2a4c-40e1-a23e-c62daab14d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:55.511437 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0bcea2-2a4c-40e1-a23e-c62daab14d34 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:55.511752 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0bcea2-2a4c-40e1-a23e-c62daab14d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:55.512120 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0bcea2-2a4c-40e1-a23e-c62daab14d34 admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:55.513030 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0bcea2-2a4c-40e1-a23e-c62daab14d34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:55.514097 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3670c0-646e-464c-a694-4f820f04e972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:55.514846 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3670c0-646e-464c-a694-4f820f04e972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:55.515122 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0bcea2-2a4c-40e1-a23e-c62daab14d34 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:55.515365 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3670c0-646e-464c-a694-4f820f04e972 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:55.515780 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3670c0-646e-464c-a694-4f820f04e972 admin admin] PATH_INFO: `/v3/users/36d31fe235d749d28f251aed98952280` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:55.517093 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3670c0-646e-464c-a694-4f820f04e972 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36d31fe235d749d28f251aed98952280)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:55.519119 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3670c0-646e-464c-a694-4f820f04e972 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:55.555473 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 894/1786] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:51:55 2022] POST /identity/v3/users => generated 283 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:51:55.556917 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0bcea2-2a4c-40e1-a23e-c62daab14d34 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:55.559041 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-1e0bcea2-2a4c-40e1-a23e-c62daab14d34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:55.586105 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-dd3670c0-646e-464c-a694-4f820f04e972 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36d31fe235d749d28f251aed98952280'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:55.586803 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-dd3670c0-646e-464c-a694-4f820f04e972 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36d31fe235d749d28f251aed98952280'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:55.588017 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-dd3670c0-646e-464c-a694-4f820f04e972 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36d31fe235d749d28f251aed98952280'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:55.608414 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-dd3670c0-646e-464c-a694-4f820f04e972 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36d31fe235d749d28f251aed98952280'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:55.618848 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-dd3670c0-646e-464c-a694-4f820f04e972 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36d31fe235d749d28f251aed98952280'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:55.646207 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 893/1787] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:51:55 2022] DELETE /identity/v3/users/36d31fe235d749d28f251aed98952280 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:55.648829 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3670c0-646e-464c-a694-4f820f04e972 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:55.656948 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-dd3670c0-646e-464c-a694-4f820f04e972 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:55.668567 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0bcea2-2a4c-40e1-a23e-c62daab14d34 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:55.736445 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bde2fa-c44c-40a8-bb83-727f2418cce1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:55.737079 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bde2fa-c44c-40a8-bb83-727f2418cce1 service cinder] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:55.737452 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bde2fa-c44c-40a8-bb83-727f2418cce1 service cinder] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:55.737993 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bde2fa-c44c-40a8-bb83-727f2418cce1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:55.740151 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-46bde2fa-c44c-40a8-bb83-727f2418cce1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:55.740797 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3670c0-646e-464c-a694-4f820f04e972 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:55.746521 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723db029-a9bc-4b69-b6ea-15390f3843aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:55.747376 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723db029-a9bc-4b69-b6ea-15390f3843aa admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:55.747768 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723db029-a9bc-4b69-b6ea-15390f3843aa admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:55.748367 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723db029-a9bc-4b69-b6ea-15390f3843aa admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:55.749155 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723db029-a9bc-4b69-b6ea-15390f3843aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:55.750860 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723db029-a9bc-4b69-b6ea-15390f3843aa admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:55.762603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 894/1788] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:55 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:51:55.766444 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723db029-a9bc-4b69-b6ea-15390f3843aa admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:55.778604 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723db029-a9bc-4b69-b6ea-15390f3843aa admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:55.782397 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:55.784334 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec2d71d-3f1b-4aa3-b868-a9d93f5b3bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:55.785038 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2d71d-3f1b-4aa3-b868-a9d93f5b3bfb admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:55.785397 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2d71d-3f1b-4aa3-b868-a9d93f5b3bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:55.785753 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2d71d-3f1b-4aa3-b868-a9d93f5b3bfb admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:55.786535 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec2d71d-3f1b-4aa3-b868-a9d93f5b3bfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:55.788472 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec2d71d-3f1b-4aa3-b868-a9d93f5b3bfb admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:55.800512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 895/1789] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 12:51:55.805205 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec2d71d-3f1b-4aa3-b868-a9d93f5b3bfb admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:55.815830 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec2d71d-3f1b-4aa3-b868-a9d93f5b3bfb admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:55.816879 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bde2fa-c44c-40a8-bb83-727f2418cce1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:55.818543 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bde2fa-c44c-40a8-bb83-727f2418cce1 service cinder] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:55.820852 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e79181-a026-4441-83d1-6a66584cb7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:55.821649 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e79181-a026-4441-83d1-6a66584cb7da admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:55.822006 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e79181-a026-4441-83d1-6a66584cb7da admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:55.822359 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e79181-a026-4441-83d1-6a66584cb7da admin admin] PATH_INFO: `/v3/projects/1cc07f98ade3464e91dc6ae904358e43/users/2333123d25ee44a48d9d924a7d9fbd12/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:55.828618 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 895/1790] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:51:55 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:51:55.838583 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e79181-a026-4441-83d1-6a66584cb7da admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cc07f98ade3464e91dc6ae904358e43, user_id=2333123d25ee44a48d9d924a7d9fbd12, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:55.841436 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e79181-a026-4441-83d1-6a66584cb7da admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:55.859616 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 896/1791] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:51:55 2022] PUT /identity/v3/projects/1cc07f98ade3464e91dc6ae904358e43/users/2333123d25ee44a48d9d924a7d9fbd12/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:51:55.871573 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34628177-8e9f-410c-8d32-b3cbbe789183 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:55.872108 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34628177-8e9f-410c-8d32-b3cbbe789183 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:55.872621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34628177-8e9f-410c-8d32-b3cbbe789183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:55.907906 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-34628177-8e9f-410c-8d32-b3cbbe789183 None None] MFA Rules not processed for user `2333123d25ee44a48d9d924a7d9fbd12`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:55.920413 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-34628177-8e9f-410c-8d32-b3cbbe789183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:56.003927 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 896/1792] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:55 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:51:56.008641 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e79181-a026-4441-83d1-6a66584cb7da admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:56.042861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:56.043370 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34628177-8e9f-410c-8d32-b3cbbe789183 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:56.044849 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-34628177-8e9f-410c-8d32-b3cbbe789183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:56.116387 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e79181-a026-4441-83d1-6a66584cb7da admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:56.121672 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ca9a4d-a5b9-4075-af83-47f2aa9a6250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:56.122246 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca9a4d-a5b9-4075-af83-47f2aa9a6250 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:56.122702 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca9a4d-a5b9-4075-af83-47f2aa9a6250 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:56.123326 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca9a4d-a5b9-4075-af83-47f2aa9a6250 admin admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:56.124060 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ca9a4d-a5b9-4075-af83-47f2aa9a6250 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:56.126149 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ca9a4d-a5b9-4075-af83-47f2aa9a6250 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:56.143283 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34628177-8e9f-410c-8d32-b3cbbe789183 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:56.148846 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c20c3-6356-4d28-92eb-d72e6553f2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:56.149524 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c20c3-6356-4d28-92eb-d72e6553f2ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:56.149806 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c20c3-6356-4d28-92eb-d72e6553f2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:56.150077 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c20c3-6356-4d28-92eb-d72e6553f2ea admin admin] PATH_INFO: `/v3/projects/9e4e02d8829643d99aaaa3d055c4df9a` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:56.151248 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c20c3-6356-4d28-92eb-d72e6553f2ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e4e02d8829643d99aaaa3d055c4df9a)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:56.153243 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c20c3-6356-4d28-92eb-d72e6553f2ea admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:56.155688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 897/1793] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:51:56 2022] POST /identity/v3/projects => generated 361 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:51:56.160113 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ca9a4d-a5b9-4075-af83-47f2aa9a6250 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:56.257887 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-231c20c3-6356-4d28-92eb-d72e6553f2ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e4e02d8829643d99aaaa3d055c4df9a', 'request_id': 'req-231c20c3-6356-4d28-92eb-d72e6553f2ea'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:56.266551 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-231c20c3-6356-4d28-92eb-d72e6553f2ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e4e02d8829643d99aaaa3d055c4df9a', 'request_id': 'req-231c20c3-6356-4d28-92eb-d72e6553f2ea'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:56.276871 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-231c20c3-6356-4d28-92eb-d72e6553f2ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e4e02d8829643d99aaaa3d055c4df9a', 'request_id': 'req-231c20c3-6356-4d28-92eb-d72e6553f2ea'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:56.281013 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:56.285029 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ca9a4d-a5b9-4075-af83-47f2aa9a6250 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:56.296352 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-231c20c3-6356-4d28-92eb-d72e6553f2ea admin admin] The token cache is being invalidate because project 9e4e02d8829643d99aaaa3d055c4df9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:51:56.296744 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-231c20c3-6356-4d28-92eb-d72e6553f2ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:56.297702 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-231c20c3-6356-4d28-92eb-d72e6553f2ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:56.299930 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 897/1794] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:51:56 2022] DELETE /identity/v3/projects/9e4e02d8829643d99aaaa3d055c4df9a => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:51:56.306013 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2751ac45-2d76-4780-8dfb-d9203e389a5b None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:56.306420 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2751ac45-2d76-4780-8dfb-d9203e389a5b None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:56.306735 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2751ac45-2d76-4780-8dfb-d9203e389a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:56.338808 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-2751ac45-2d76-4780-8dfb-d9203e389a5b None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:56.341051 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-2751ac45-2d76-4780-8dfb-d9203e389a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:56.360737 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b977e2-8d84-4821-8d63-8cd5a4c9c061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:56.361375 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b977e2-8d84-4821-8d63-8cd5a4c9c061 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:56.361770 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b977e2-8d84-4821-8d63-8cd5a4c9c061 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:56.362047 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b977e2-8d84-4821-8d63-8cd5a4c9c061 admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:56.362728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b977e2-8d84-4821-8d63-8cd5a4c9c061 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:56.364741 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b977e2-8d84-4821-8d63-8cd5a4c9c061 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:56.392049 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 898/1795] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:51:56 2022] POST /identity/v3/users => generated 287 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:51:56.393262 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b977e2-8d84-4821-8d63-8cd5a4c9c061 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:56.395122 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-82b977e2-8d84-4821-8d63-8cd5a4c9c061 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:56.408065 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b977e2-8d84-4821-8d63-8cd5a4c9c061 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:56.413879 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5ff56b-0f87-404c-8f8a-afbb95b9763d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:56.414649 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ff56b-0f87-404c-8f8a-afbb95b9763d admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:56.415004 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ff56b-0f87-404c-8f8a-afbb95b9763d admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:56.415287 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ff56b-0f87-404c-8f8a-afbb95b9763d admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:56.416038 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5ff56b-0f87-404c-8f8a-afbb95b9763d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:56.416139 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 898/1796] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:56.417864 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5ff56b-0f87-404c-8f8a-afbb95b9763d admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:56.424093 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d798435-c35e-455e-9a23-16dd61867687 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:56.424644 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d798435-c35e-455e-9a23-16dd61867687 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:56.425022 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d798435-c35e-455e-9a23-16dd61867687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:56.429982 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 899/1797] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 12:51:56.433924 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5ff56b-0f87-404c-8f8a-afbb95b9763d admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:56.447422 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5ff56b-0f87-404c-8f8a-afbb95b9763d admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:56.450171 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-2d798435-c35e-455e-9a23-16dd61867687 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:56.451369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-2d798435-c35e-455e-9a23-16dd61867687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:56.452862 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ea01a-8ef4-4a52-b791-8a1a5a6da9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:56.453626 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ea01a-8ef4-4a52-b791-8a1a5a6da9a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:56.454023 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ea01a-8ef4-4a52-b791-8a1a5a6da9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:56.454396 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ea01a-8ef4-4a52-b791-8a1a5a6da9a4 admin admin] PATH_INFO: `/v3/projects/8ba249f0d79f45dab8d43bcf6dc71868/users/7c4a230a981c4e629b68c97d3f30830b/roles/0e78444e43c744668178971af4087ca0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:56.460465 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 899/1798] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:51:56.464623 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d798435-c35e-455e-9a23-16dd61867687 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:56.472138 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8ea01a-8ef4-4a52-b791-8a1a5a6da9a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ba249f0d79f45dab8d43bcf6dc71868, user_id=7c4a230a981c4e629b68c97d3f30830b, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:56.475536 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8ea01a-8ef4-4a52-b791-8a1a5a6da9a4 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:56.476651 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d798435-c35e-455e-9a23-16dd61867687 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:56.484311 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa520b87-1e16-4e15-ba81-10dbe6c4613e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:56.485073 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa520b87-1e16-4e15-ba81-10dbe6c4613e admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:56.485447 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa520b87-1e16-4e15-ba81-10dbe6c4613e admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:56.485836 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa520b87-1e16-4e15-ba81-10dbe6c4613e admin admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:56.486658 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa520b87-1e16-4e15-ba81-10dbe6c4613e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:51:56.487049 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa520b87-1e16-4e15-ba81-10dbe6c4613e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:56.488882 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa520b87-1e16-4e15-ba81-10dbe6c4613e admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:56.490381 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 900/1799] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:51:56 2022] PUT /identity/v3/projects/8ba249f0d79f45dab8d43bcf6dc71868/users/7c4a230a981c4e629b68c97d3f30830b/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:51:56.494767 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ea01a-8ef4-4a52-b791-8a1a5a6da9a4 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:56.504447 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 900/1800] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:51:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:51:56.505880 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:56.509500 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa520b87-1e16-4e15-ba81-10dbe6c4613e admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:56.523856 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa520b87-1e16-4e15-ba81-10dbe6c4613e admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:56.574314 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ea01a-8ef4-4a52-b791-8a1a5a6da9a4 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:56.579857 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60953853-fc29-4469-af33-81d82c1d9fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:56.580631 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60953853-fc29-4469-af33-81d82c1d9fae admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:56.581179 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60953853-fc29-4469-af33-81d82c1d9fae admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:56.581525 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60953853-fc29-4469-af33-81d82c1d9fae admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:56.582314 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60953853-fc29-4469-af33-81d82c1d9fae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:56.584033 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60953853-fc29-4469-af33-81d82c1d9fae admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:56.593044 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa884e7-c1e9-4f7e-8920-e7f77e1b90d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:56.593723 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa884e7-c1e9-4f7e-8920-e7f77e1b90d8 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:56.594031 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa884e7-c1e9-4f7e-8920-e7f77e1b90d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:56.594333 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa884e7-c1e9-4f7e-8920-e7f77e1b90d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:56.595031 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa884e7-c1e9-4f7e-8920-e7f77e1b90d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:56.597038 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa884e7-c1e9-4f7e-8920-e7f77e1b90d8 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:56.599671 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 901/1801] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 12:51:56.602622 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60953853-fc29-4469-af33-81d82c1d9fae admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:56.612721 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60953853-fc29-4469-af33-81d82c1d9fae admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:56.617492 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1565e22-3684-4a0c-ac07-e2d4171dbb93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:56.618306 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1565e22-3684-4a0c-ac07-e2d4171dbb93 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:56.618588 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1565e22-3684-4a0c-ac07-e2d4171dbb93 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:56.618866 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1565e22-3684-4a0c-ac07-e2d4171dbb93 admin admin] PATH_INFO: `/v3/projects/8ba249f0d79f45dab8d43bcf6dc71868/users/7c4a230a981c4e629b68c97d3f30830b/roles/0e78444e43c744668178971af4087ca0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:56.621032 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1565e22-3684-4a0c-ac07-e2d4171dbb93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ba249f0d79f45dab8d43bcf6dc71868, user_id=7c4a230a981c4e629b68c97d3f30830b, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:56.624307 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1565e22-3684-4a0c-ac07-e2d4171dbb93 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:56.630561 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 901/1802] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:51:56 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:51:56.632277 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa884e7-c1e9-4f7e-8920-e7f77e1b90d8 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:56.636554 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 902/1803] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:51:56 2022] PUT /identity/v3/projects/8ba249f0d79f45dab8d43bcf6dc71868/users/7c4a230a981c4e629b68c97d3f30830b/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:51:56.647547 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94aa651-b67a-44ba-9fb5-92b129469b86 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:56.647832 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94aa651-b67a-44ba-9fb5-92b129469b86 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:56.648130 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94aa651-b67a-44ba-9fb5-92b129469b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:56.685552 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-e94aa651-b67a-44ba-9fb5-92b129469b86 None None] MFA Rules not processed for user `7c4a230a981c4e629b68c97d3f30830b`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:56.686896 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-e94aa651-b67a-44ba-9fb5-92b129469b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:56.713169 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:56.717865 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa884e7-c1e9-4f7e-8920-e7f77e1b90d8 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:56.772523 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 903/1804] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:56 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:51:56.786677 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94aa651-b67a-44ba-9fb5-92b129469b86 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:56.789338 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-e94aa651-b67a-44ba-9fb5-92b129469b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:56.800400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344def7e-08d3-4b26-8224-7628c00aa510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:56.801140 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344def7e-08d3-4b26-8224-7628c00aa510 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:56.801501 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344def7e-08d3-4b26-8224-7628c00aa510 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:56.801849 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344def7e-08d3-4b26-8224-7628c00aa510 admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:56.802602 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344def7e-08d3-4b26-8224-7628c00aa510 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:56.804890 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344def7e-08d3-4b26-8224-7628c00aa510 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:56.832239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 902/1805] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:51:56 2022] POST /identity/v3/users => generated 293 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:51:56.836746 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344def7e-08d3-4b26-8224-7628c00aa510 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:56.853446 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344def7e-08d3-4b26-8224-7628c00aa510 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:56.857891 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94aa651-b67a-44ba-9fb5-92b129469b86 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:56.860394 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c1f86a-6de2-4a3d-a636-2141aa410e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:56.861181 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1f86a-6de2-4a3d-a636-2141aa410e48 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:56.861537 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1f86a-6de2-4a3d-a636-2141aa410e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:56.861881 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1f86a-6de2-4a3d-a636-2141aa410e48 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:56.862669 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c1f86a-6de2-4a3d-a636-2141aa410e48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:56.862790 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b90586b-7531-401d-93cd-36aee83c0af6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:56.863458 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b90586b-7531-401d-93cd-36aee83c0af6 service cinder] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:56.863826 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b90586b-7531-401d-93cd-36aee83c0af6 service cinder] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:56.864162 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b90586b-7531-401d-93cd-36aee83c0af6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:56.864658 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c1f86a-6de2-4a3d-a636-2141aa410e48 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:56.874705 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b90586b-7531-401d-93cd-36aee83c0af6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:56.878823 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 903/1806] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:51:56.878964 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b90586b-7531-401d-93cd-36aee83c0af6 service cinder] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:56.879893 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c1f86a-6de2-4a3d-a636-2141aa410e48 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:56.890525 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c1f86a-6de2-4a3d-a636-2141aa410e48 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:56.896795 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2325d82c-6370-4c99-853e-badb787c17ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:56.898010 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325d82c-6370-4c99-853e-badb787c17ca admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:56.898310 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325d82c-6370-4c99-853e-badb787c17ca admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:56.898650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325d82c-6370-4c99-853e-badb787c17ca admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:56.899413 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2325d82c-6370-4c99-853e-badb787c17ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:56.901091 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2325d82c-6370-4c99-853e-badb787c17ca admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:56.904145 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 904/1807] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:51:56 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:51:56.914851 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 904/1808] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 12:51:56.920005 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b90586b-7531-401d-93cd-36aee83c0af6 service cinder] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:56.940630 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b90586b-7531-401d-93cd-36aee83c0af6 service cinder] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:56.955682 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f4d19-2550-4307-804e-045726e07273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:56.956636 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f4d19-2550-4307-804e-045726e07273 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:56.956924 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f4d19-2550-4307-804e-045726e07273 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:56.957275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f4d19-2550-4307-804e-045726e07273 admin admin] PATH_INFO: `/v3/projects/4238f28915854f2c9a7ef86390031b11/users/242f1e530cd74a3696805098d8802f83/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:56.980650 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186f4d19-2550-4307-804e-045726e07273 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4238f28915854f2c9a7ef86390031b11, user_id=242f1e530cd74a3696805098d8802f83, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:56.983871 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186f4d19-2550-4307-804e-045726e07273 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:56.987519 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:57.000165 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 905/1809] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:51:56 2022] PUT /identity/v3/projects/4238f28915854f2c9a7ef86390031b11/users/242f1e530cd74a3696805098d8802f83/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:57.010658 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22db1f-27d0-4317-a178-646aa28dd8c5 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:57.011261 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22db1f-27d0-4317-a178-646aa28dd8c5 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:57.011802 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22db1f-27d0-4317-a178-646aa28dd8c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:57.014493 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f4d19-2550-4307-804e-045726e07273 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:57.054537 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-8f22db1f-27d0-4317-a178-646aa28dd8c5 None None] MFA Rules not processed for user `242f1e530cd74a3696805098d8802f83`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:57.055844 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-8f22db1f-27d0-4317-a178-646aa28dd8c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:57.111913 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f4d19-2550-4307-804e-045726e07273 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:57.122091 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6885b3-7e72-4eef-825f-6885f270af82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:57.123150 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6885b3-7e72-4eef-825f-6885f270af82 service cinder] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:57.123661 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6885b3-7e72-4eef-825f-6885f270af82 service cinder] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:57.123961 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6885b3-7e72-4eef-825f-6885f270af82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:57.126248 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-4a6885b3-7e72-4eef-825f-6885f270af82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:57.171423 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 905/1810] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:57.172614 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22db1f-27d0-4317-a178-646aa28dd8c5 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:57.246957 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6885b3-7e72-4eef-825f-6885f270af82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:57.249188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6885b3-7e72-4eef-825f-6885f270af82 service cinder] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:57.264426 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 906/1811] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:51:57 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 12:51:57.312033 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22db1f-27d0-4317-a178-646aa28dd8c5 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:57.318649 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4845d63f-6f08-4f0d-aa7c-85c75aacf0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:57.319414 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845d63f-6f08-4f0d-aa7c-85c75aacf0f3 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:57.319798 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845d63f-6f08-4f0d-aa7c-85c75aacf0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:57.320142 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845d63f-6f08-4f0d-aa7c-85c75aacf0f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:57.320953 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4845d63f-6f08-4f0d-aa7c-85c75aacf0f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:57.324291 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4845d63f-6f08-4f0d-aa7c-85c75aacf0f3 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:57.331517 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:57.369073 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 906/1812] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:51:57 2022] POST /identity/v3/projects => generated 385 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:51:57.373662 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6885b3-7e72-4eef-825f-6885f270af82 service cinder] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:57.390899 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6885b3-7e72-4eef-825f-6885f270af82 service cinder] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:57.479001 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cbf685-af2d-4e14-a0d5-24d23c845df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:57.479748 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cbf685-af2d-4e14-a0d5-24d23c845df1 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:57.480038 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cbf685-af2d-4e14-a0d5-24d23c845df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:57.480316 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cbf685-af2d-4e14-a0d5-24d23c845df1 admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:57.481007 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cbf685-af2d-4e14-a0d5-24d23c845df1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:57.482980 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cbf685-af2d-4e14-a0d5-24d23c845df1 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:57.507372 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 907/1813] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:51:57 2022] POST /identity/v3/users => generated 299 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:51:57.511520 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4845d63f-6f08-4f0d-aa7c-85c75aacf0f3 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:57.525161 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4845d63f-6f08-4f0d-aa7c-85c75aacf0f3 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:57.532459 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc52d771-9a1b-427e-a52c-e7443cd87bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:57.533273 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc52d771-9a1b-427e-a52c-e7443cd87bcf admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:57.533754 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc52d771-9a1b-427e-a52c-e7443cd87bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:57.535022 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc52d771-9a1b-427e-a52c-e7443cd87bcf admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:57.535022 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc52d771-9a1b-427e-a52c-e7443cd87bcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:57.536727 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc52d771-9a1b-427e-a52c-e7443cd87bcf admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:57.549826 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 907/1814] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:51:57.553966 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cbf685-af2d-4e14-a0d5-24d23c845df1 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:57.564758 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cbf685-af2d-4e14-a0d5-24d23c845df1 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:57.570459 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d70b8d-5d66-43f7-897b-03bd8e9e8787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:57.571269 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d70b8d-5d66-43f7-897b-03bd8e9e8787 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:57.571686 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d70b8d-5d66-43f7-897b-03bd8e9e8787 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:57.572149 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d70b8d-5d66-43f7-897b-03bd8e9e8787 admin admin] PATH_INFO: `/v3/projects/bd5bbbd46ba6471cb838640f3819ec7c/users/14b76b832cd840da9a58ab2395a6ac8c/roles/0e78444e43c744668178971af4087ca0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:57.591165 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d70b8d-5d66-43f7-897b-03bd8e9e8787 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd5bbbd46ba6471cb838640f3819ec7c, user_id=14b76b832cd840da9a58ab2395a6ac8c, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:57.594390 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d70b8d-5d66-43f7-897b-03bd8e9e8787 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:57.618373 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 908/1815] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:51:57 2022] PUT /identity/v3/projects/bd5bbbd46ba6471cb838640f3819ec7c/users/14b76b832cd840da9a58ab2395a6ac8c/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:51:57.622502 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc52d771-9a1b-427e-a52c-e7443cd87bcf admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:57.650246 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc52d771-9a1b-427e-a52c-e7443cd87bcf admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:57.786452 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb439a84-0938-460e-9fc0-ecac9d88baca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:57.787087 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb439a84-0938-460e-9fc0-ecac9d88baca admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:57.787861 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb439a84-0938-460e-9fc0-ecac9d88baca admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:57.788137 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb439a84-0938-460e-9fc0-ecac9d88baca admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:57.789163 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb439a84-0938-460e-9fc0-ecac9d88baca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:57.790866 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb439a84-0938-460e-9fc0-ecac9d88baca admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:57.802872 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 908/1816] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:57 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:51:57.807565 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d70b8d-5d66-43f7-897b-03bd8e9e8787 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:57.822332 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d70b8d-5d66-43f7-897b-03bd8e9e8787 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:57.828024 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b4a51f-4144-4088-a88a-51a226a811b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:57.828772 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b4a51f-4144-4088-a88a-51a226a811b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:57.829239 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b4a51f-4144-4088-a88a-51a226a811b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:57.829600 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b4a51f-4144-4088-a88a-51a226a811b7 admin admin] PATH_INFO: `/v3/projects/bd5bbbd46ba6471cb838640f3819ec7c/users/14b76b832cd840da9a58ab2395a6ac8c/roles/0e78444e43c744668178971af4087ca0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:57.831668 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b4a51f-4144-4088-a88a-51a226a811b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd5bbbd46ba6471cb838640f3819ec7c, user_id=14b76b832cd840da9a58ab2395a6ac8c, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:57.834917 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b4a51f-4144-4088-a88a-51a226a811b7 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:57.846300 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 909/1817] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:51:57 2022] PUT /identity/v3/projects/bd5bbbd46ba6471cb838640f3819ec7c/users/14b76b832cd840da9a58ab2395a6ac8c/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:57.862121 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8395e2d-eb46-4639-92c7-a4aef7d4a5cd None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:57.862635 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8395e2d-eb46-4639-92c7-a4aef7d4a5cd None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:57.862744 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8395e2d-eb46-4639-92c7-a4aef7d4a5cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:57.895150 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-b8395e2d-eb46-4639-92c7-a4aef7d4a5cd None None] MFA Rules not processed for user `14b76b832cd840da9a58ab2395a6ac8c`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:57.896473 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-b8395e2d-eb46-4639-92c7-a4aef7d4a5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:58.017568 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 909/1818] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:57 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:58.031267 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b4a51f-4144-4088-a88a-51a226a811b7 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:58.036805 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:58.113613 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b4a51f-4144-4088-a88a-51a226a811b7 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:58.118052 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c10d85-2d25-4beb-8e9f-4fb009606f2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:58.118674 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c10d85-2d25-4beb-8e9f-4fb009606f2c service cinder] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:58.118959 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c10d85-2d25-4beb-8e9f-4fb009606f2c service cinder] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:58.119263 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c10d85-2d25-4beb-8e9f-4fb009606f2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:58.129262 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c10d85-2d25-4beb-8e9f-4fb009606f2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:58.130730 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c10d85-2d25-4beb-8e9f-4fb009606f2c service cinder] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:58.140154 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 910/1819] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:51:58 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:51:58.168606 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8395e2d-eb46-4639-92c7-a4aef7d4a5cd None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:58.229631 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:58.234058 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8395e2d-eb46-4639-92c7-a4aef7d4a5cd None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:58.239836 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b609ab67-8c9e-4db3-864e-10eceb39f5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:58.241215 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b609ab67-8c9e-4db3-864e-10eceb39f5b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:58.241496 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b609ab67-8c9e-4db3-864e-10eceb39f5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:58.241773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b609ab67-8c9e-4db3-864e-10eceb39f5b4 admin admin] PATH_INFO: `/v3/users/242f1e530cd74a3696805098d8802f83` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:58.242960 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b609ab67-8c9e-4db3-864e-10eceb39f5b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=242f1e530cd74a3696805098d8802f83)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:58.244957 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b609ab67-8c9e-4db3-864e-10eceb39f5b4 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:58.293541 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-b609ab67-8c9e-4db3-864e-10eceb39f5b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '242f1e530cd74a3696805098d8802f83'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:58.301580 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-b609ab67-8c9e-4db3-864e-10eceb39f5b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '242f1e530cd74a3696805098d8802f83'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:58.302338 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-b609ab67-8c9e-4db3-864e-10eceb39f5b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '242f1e530cd74a3696805098d8802f83'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:58.309658 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-b609ab67-8c9e-4db3-864e-10eceb39f5b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '242f1e530cd74a3696805098d8802f83'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:58.322596 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-b609ab67-8c9e-4db3-864e-10eceb39f5b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '242f1e530cd74a3696805098d8802f83'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:58.328554 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 910/1820] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:51:58 2022] DELETE /identity/v3/users/242f1e530cd74a3696805098d8802f83 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:58.333261 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955876df-1d4f-40d5-894f-304bd64ff95a None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:58.333554 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955876df-1d4f-40d5-894f-304bd64ff95a None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:58.333828 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955876df-1d4f-40d5-894f-304bd64ff95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:58.353356 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-955876df-1d4f-40d5-894f-304bd64ff95a None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:58.354577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-955876df-1d4f-40d5-894f-304bd64ff95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:58.418259 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 911/1821] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:58.428331 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b609ab67-8c9e-4db3-864e-10eceb39f5b4 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:58.429903 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-b609ab67-8c9e-4db3-864e-10eceb39f5b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:58.492456 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b609ab67-8c9e-4db3-864e-10eceb39f5b4 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:58.496182 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5978c34-aead-4dd0-a357-fed0ed2395dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:58.496799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5978c34-aead-4dd0-a357-fed0ed2395dc service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:58.497081 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5978c34-aead-4dd0-a357-fed0ed2395dc service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:58.497356 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5978c34-aead-4dd0-a357-fed0ed2395dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:58.507528 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5978c34-aead-4dd0-a357-fed0ed2395dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:58.509129 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5978c34-aead-4dd0-a357-fed0ed2395dc service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:58.518797 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 911/1822] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:51:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:51:58.785247 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955876df-1d4f-40d5-894f-304bd64ff95a None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:58.787204 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-955876df-1d4f-40d5-894f-304bd64ff95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:58.801338 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955876df-1d4f-40d5-894f-304bd64ff95a None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:58.806895 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2ef68-6351-4704-bd30-c9a48822c8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:58.807545 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2ef68-6351-4704-bd30-c9a48822c8fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:58.807857 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2ef68-6351-4704-bd30-c9a48822c8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:58.808129 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2ef68-6351-4704-bd30-c9a48822c8fc admin admin] PATH_INFO: `/v3/projects/4238f28915854f2c9a7ef86390031b11` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:58.809295 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2ef68-6351-4704-bd30-c9a48822c8fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=4238f28915854f2c9a7ef86390031b11)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:58.811421 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2ef68-6351-4704-bd30-c9a48822c8fc admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:58.867163 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-a9b2ef68-6351-4704-bd30-c9a48822c8fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4238f28915854f2c9a7ef86390031b11', 'request_id': 'req-a9b2ef68-6351-4704-bd30-c9a48822c8fc'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:58.875548 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-a9b2ef68-6351-4704-bd30-c9a48822c8fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4238f28915854f2c9a7ef86390031b11', 'request_id': 'req-a9b2ef68-6351-4704-bd30-c9a48822c8fc'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:58.883690 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-a9b2ef68-6351-4704-bd30-c9a48822c8fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4238f28915854f2c9a7ef86390031b11', 'request_id': 'req-a9b2ef68-6351-4704-bd30-c9a48822c8fc'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:58.895078 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-a9b2ef68-6351-4704-bd30-c9a48822c8fc admin admin] The token cache is being invalidate because project 4238f28915854f2c9a7ef86390031b11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:51:58.895369 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-a9b2ef68-6351-4704-bd30-c9a48822c8fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:58.896108 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-a9b2ef68-6351-4704-bd30-c9a48822c8fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:58.897945 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 912/1823] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:51:58 2022] DELETE /identity/v3/projects/4238f28915854f2c9a7ef86390031b11 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:58.902122 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5978c34-aead-4dd0-a357-fed0ed2395dc service neutron] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:58.913047 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5978c34-aead-4dd0-a357-fed0ed2395dc service neutron] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:58.984475 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ed6beb-04f2-477d-abc6-315e84d55e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:58.985227 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed6beb-04f2-477d-abc6-315e84d55e09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:58.985603 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed6beb-04f2-477d-abc6-315e84d55e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:58.985972 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed6beb-04f2-477d-abc6-315e84d55e09 admin admin] PATH_INFO: `/v3/users/14b76b832cd840da9a58ab2395a6ac8c` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:58.987409 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ed6beb-04f2-477d-abc6-315e84d55e09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14b76b832cd840da9a58ab2395a6ac8c)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:58.991911 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ed6beb-04f2-477d-abc6-315e84d55e09 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:59.021722 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:59.053268 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-32ed6beb-04f2-477d-abc6-315e84d55e09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14b76b832cd840da9a58ab2395a6ac8c'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:59.060852 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-32ed6beb-04f2-477d-abc6-315e84d55e09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14b76b832cd840da9a58ab2395a6ac8c'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:59.061627 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-32ed6beb-04f2-477d-abc6-315e84d55e09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14b76b832cd840da9a58ab2395a6ac8c'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:59.070626 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-32ed6beb-04f2-477d-abc6-315e84d55e09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14b76b832cd840da9a58ab2395a6ac8c'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:59.082661 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-32ed6beb-04f2-477d-abc6-315e84d55e09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14b76b832cd840da9a58ab2395a6ac8c'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:59.089377 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 912/1824] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:51:58 2022] DELETE /identity/v3/users/14b76b832cd840da9a58ab2395a6ac8c => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:59.458928 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2ef68-6351-4704-bd30-c9a48822c8fc admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:59.460514 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-a9b2ef68-6351-4704-bd30-c9a48822c8fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:59.476131 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2ef68-6351-4704-bd30-c9a48822c8fc admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:59.512938 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:51:59.547899 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c855d3a1-1dab-4cef-8658-6d34f7b8973b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:59.548652 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855d3a1-1dab-4cef-8658-6d34f7b8973b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:59.549019 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855d3a1-1dab-4cef-8658-6d34f7b8973b admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:59.549353 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855d3a1-1dab-4cef-8658-6d34f7b8973b admin admin] PATH_INFO: `/v3/projects/bd5bbbd46ba6471cb838640f3819ec7c` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:59.550594 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c855d3a1-1dab-4cef-8658-6d34f7b8973b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd5bbbd46ba6471cb838640f3819ec7c)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:59.552621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c855d3a1-1dab-4cef-8658-6d34f7b8973b admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:59.602977 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-c855d3a1-1dab-4cef-8658-6d34f7b8973b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd5bbbd46ba6471cb838640f3819ec7c', 'request_id': 'req-c855d3a1-1dab-4cef-8658-6d34f7b8973b'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:59.609604 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-c855d3a1-1dab-4cef-8658-6d34f7b8973b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd5bbbd46ba6471cb838640f3819ec7c', 'request_id': 'req-c855d3a1-1dab-4cef-8658-6d34f7b8973b'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:59.617147 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-c855d3a1-1dab-4cef-8658-6d34f7b8973b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd5bbbd46ba6471cb838640f3819ec7c', 'request_id': 'req-c855d3a1-1dab-4cef-8658-6d34f7b8973b'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:59.628502 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-c855d3a1-1dab-4cef-8658-6d34f7b8973b admin admin] The token cache is being invalidate because project bd5bbbd46ba6471cb838640f3819ec7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:51:59.628792 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-c855d3a1-1dab-4cef-8658-6d34f7b8973b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:59.629445 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-c855d3a1-1dab-4cef-8658-6d34f7b8973b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:51:59.631497 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 913/1825] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:51:59 2022] DELETE /identity/v3/projects/bd5bbbd46ba6471cb838640f3819ec7c => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:51:59.636586 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb3669-046a-45b6-90d7-bccbbf576469 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:59.636879 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb3669-046a-45b6-90d7-bccbbf576469 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:59.637181 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb3669-046a-45b6-90d7-bccbbf576469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:59.658231 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-e9eb3669-046a-45b6-90d7-bccbbf576469 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:59.659436 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-e9eb3669-046a-45b6-90d7-bccbbf576469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:59.720562 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 913/1826] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:51:59.728124 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a299b-1b5c-4fdd-b944-2ed401ebbff1 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:59.728410 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a299b-1b5c-4fdd-b944-2ed401ebbff1 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:59.728719 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a299b-1b5c-4fdd-b944-2ed401ebbff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:59.749134 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-638a299b-1b5c-4fdd-b944-2ed401ebbff1 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:51:59.750340 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-638a299b-1b5c-4fdd-b944-2ed401ebbff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:51:59.759005 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 914/1827] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:51:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:51:59.763330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb3669-046a-45b6-90d7-bccbbf576469 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:59.773835 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb3669-046a-45b6-90d7-bccbbf576469 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:59.778561 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89927fb8-3b4b-4ad8-ae54-50b865d5a7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:59.779177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89927fb8-3b4b-4ad8-ae54-50b865d5a7a2 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:59.779459 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89927fb8-3b4b-4ad8-ae54-50b865d5a7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:59.779765 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89927fb8-3b4b-4ad8-ae54-50b865d5a7a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:59.780469 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89927fb8-3b4b-4ad8-ae54-50b865d5a7a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:51:59.780773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89927fb8-3b4b-4ad8-ae54-50b865d5a7a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:59.782367 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89927fb8-3b4b-4ad8-ae54-50b865d5a7a2 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:59.794723 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 914/1828] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:51:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:51:59.798991 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638a299b-1b5c-4fdd-b944-2ed401ebbff1 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:59.808315 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638a299b-1b5c-4fdd-b944-2ed401ebbff1 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:59.812956 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265fb240-c7f9-4f50-9357-69c34a3d04b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:59.813550 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265fb240-c7f9-4f50-9357-69c34a3d04b2 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:59.813830 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265fb240-c7f9-4f50-9357-69c34a3d04b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:59.814101 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265fb240-c7f9-4f50-9357-69c34a3d04b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:59.814760 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265fb240-c7f9-4f50-9357-69c34a3d04b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:59.816724 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265fb240-c7f9-4f50-9357-69c34a3d04b2 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:59.846436 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 915/1829] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:51:59 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:51:59.850515 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89927fb8-3b4b-4ad8-ae54-50b865d5a7a2 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:59.860599 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89927fb8-3b4b-4ad8-ae54-50b865d5a7a2 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:59.919985 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75752d9e-5a40-40db-8122-3aaa05d85239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:59.920611 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75752d9e-5a40-40db-8122-3aaa05d85239 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:59.920893 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75752d9e-5a40-40db-8122-3aaa05d85239 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:59.921166 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75752d9e-5a40-40db-8122-3aaa05d85239 admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:59.922003 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75752d9e-5a40-40db-8122-3aaa05d85239 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:59.923996 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75752d9e-5a40-40db-8122-3aaa05d85239 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:59.951599 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 915/1830] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:51:59 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:51:59.953027 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265fb240-c7f9-4f50-9357-69c34a3d04b2 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:51:59.965859 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265fb240-c7f9-4f50-9357-69c34a3d04b2 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:51:59.970657 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b3695d-c917-467c-a54d-50d577057dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:51:59.971213 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b3695d-c917-467c-a54d-50d577057dc4 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:51:59.971523 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b3695d-c917-467c-a54d-50d577057dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:51:59.971843 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b3695d-c917-467c-a54d-50d577057dc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:51:59.972556 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b3695d-c917-467c-a54d-50d577057dc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:51:59.974172 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b3695d-c917-467c-a54d-50d577057dc4 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:51:59.984637 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 916/1831] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:51:59.988807 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75752d9e-5a40-40db-8122-3aaa05d85239 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:00.001981 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75752d9e-5a40-40db-8122-3aaa05d85239 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:00.009376 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c574aa9-a89f-4e2e-9235-c0412bcebe21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:00.010140 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c574aa9-a89f-4e2e-9235-c0412bcebe21 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:00.010431 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c574aa9-a89f-4e2e-9235-c0412bcebe21 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:00.010711 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c574aa9-a89f-4e2e-9235-c0412bcebe21 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:00.011425 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c574aa9-a89f-4e2e-9235-c0412bcebe21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:00.013089 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c574aa9-a89f-4e2e-9235-c0412bcebe21 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:00.026353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 916/1832] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 12:52:00.030713 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b3695d-c917-467c-a54d-50d577057dc4 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:00.043739 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b3695d-c917-467c-a54d-50d577057dc4 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:00.048946 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e6849a-5086-494b-b36c-20cec8e1521e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:00.049623 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e6849a-5086-494b-b36c-20cec8e1521e admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:00.049911 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e6849a-5086-494b-b36c-20cec8e1521e admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:00.050193 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e6849a-5086-494b-b36c-20cec8e1521e admin admin] PATH_INFO: `/v3/projects/f8702a1ddd484978965e616131dff4f3/users/0dbdc88d8afd423da8b86ff5bf51494e/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:00.071799 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e6849a-5086-494b-b36c-20cec8e1521e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8702a1ddd484978965e616131dff4f3, user_id=0dbdc88d8afd423da8b86ff5bf51494e, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:00.076818 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e6849a-5086-494b-b36c-20cec8e1521e admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:00.111276 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 917/1833] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:52:00 2022] PUT /identity/v3/projects/f8702a1ddd484978965e616131dff4f3/users/0dbdc88d8afd423da8b86ff5bf51494e/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:00.121405 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3ef46-12e2-4769-9b56-f14b7a7d70a5 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:00.121716 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3ef46-12e2-4769-9b56-f14b7a7d70a5 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:00.121987 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3ef46-12e2-4769-9b56-f14b7a7d70a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:00.130724 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:00.156769 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-07e3ef46-12e2-4769-9b56-f14b7a7d70a5 None None] MFA Rules not processed for user `0dbdc88d8afd423da8b86ff5bf51494e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:00.158065 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-07e3ef46-12e2-4769-9b56-f14b7a7d70a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:00.278427 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 917/1834] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:00.282875 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e6849a-5086-494b-b36c-20cec8e1521e admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:00.367665 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:00.374458 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e6849a-5086-494b-b36c-20cec8e1521e admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:00.381941 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76769d72-7b64-4804-8a92-34a912e8797a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:00.382966 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76769d72-7b64-4804-8a92-34a912e8797a admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:00.383801 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76769d72-7b64-4804-8a92-34a912e8797a admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:00.384538 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76769d72-7b64-4804-8a92-34a912e8797a admin admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:00.385371 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76769d72-7b64-4804-8a92-34a912e8797a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:00.387760 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76769d72-7b64-4804-8a92-34a912e8797a admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:00.433696 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 918/1835] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:52:00 2022] POST /identity/v3/projects => generated 385 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:52:00.434949 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3ef46-12e2-4769-9b56-f14b7a7d70a5 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:00.450582 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3ef46-12e2-4769-9b56-f14b7a7d70a5 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:00.537019 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098eeaa1-4cea-493d-92c3-b719205de346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:00.538475 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098eeaa1-4cea-493d-92c3-b719205de346 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:00.539116 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098eeaa1-4cea-493d-92c3-b719205de346 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:00.539758 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098eeaa1-4cea-493d-92c3-b719205de346 admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:00.540744 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098eeaa1-4cea-493d-92c3-b719205de346 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:00.543249 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098eeaa1-4cea-493d-92c3-b719205de346 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:00.576301 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 918/1836] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:52:00 2022] POST /identity/v3/users => generated 299 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:52:00.580216 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76769d72-7b64-4804-8a92-34a912e8797a admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:00.590759 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76769d72-7b64-4804-8a92-34a912e8797a admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:00.597404 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18a5f2f-092d-4bfd-894e-8d12a749c269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:00.598095 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a5f2f-092d-4bfd-894e-8d12a749c269 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:00.598409 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a5f2f-092d-4bfd-894e-8d12a749c269 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:00.598714 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a5f2f-092d-4bfd-894e-8d12a749c269 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:00.599479 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18a5f2f-092d-4bfd-894e-8d12a749c269 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:00.601846 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18a5f2f-092d-4bfd-894e-8d12a749c269 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:00.613483 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 919/1837] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:52:00.618386 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098eeaa1-4cea-493d-92c3-b719205de346 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:00.629986 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098eeaa1-4cea-493d-92c3-b719205de346 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:00.634953 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e2b128-56d9-48bb-8b59-29f151c4d4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:00.635708 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2b128-56d9-48bb-8b59-29f151c4d4c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:00.635990 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2b128-56d9-48bb-8b59-29f151c4d4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:00.636258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2b128-56d9-48bb-8b59-29f151c4d4c8 admin admin] PATH_INFO: `/v3/projects/c32ba0a380a54b69a68a162a0194b45c/users/5356618994a14b1ebab8f28f60a5a04d/roles/0e78444e43c744668178971af4087ca0` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:00.658799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e2b128-56d9-48bb-8b59-29f151c4d4c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c32ba0a380a54b69a68a162a0194b45c, user_id=5356618994a14b1ebab8f28f60a5a04d, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:00.662762 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e2b128-56d9-48bb-8b59-29f151c4d4c8 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:00.681093 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 919/1838] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:52:00 2022] PUT /identity/v3/projects/c32ba0a380a54b69a68a162a0194b45c/users/5356618994a14b1ebab8f28f60a5a04d/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:00.682257 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18a5f2f-092d-4bfd-894e-8d12a749c269 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:00.695990 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18a5f2f-092d-4bfd-894e-8d12a749c269 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:00.761610 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0306ac-326e-41f6-a2c1-587b22240406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:00.762249 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0306ac-326e-41f6-a2c1-587b22240406 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:00.762529 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0306ac-326e-41f6-a2c1-587b22240406 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:00.762824 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0306ac-326e-41f6-a2c1-587b22240406 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:00.763506 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0306ac-326e-41f6-a2c1-587b22240406 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:00.765233 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0306ac-326e-41f6-a2c1-587b22240406 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:00.781632 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 920/1839] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:00 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 12:52:00.785997 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e2b128-56d9-48bb-8b59-29f151c4d4c8 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:00.800367 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e2b128-56d9-48bb-8b59-29f151c4d4c8 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:00.807161 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc8c8fd-3e29-434a-ac45-d7e1435fd62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:00.807753 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc8c8fd-3e29-434a-ac45-d7e1435fd62b admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:00.808036 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc8c8fd-3e29-434a-ac45-d7e1435fd62b admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:00.808315 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc8c8fd-3e29-434a-ac45-d7e1435fd62b admin admin] PATH_INFO: `/v3/projects/c32ba0a380a54b69a68a162a0194b45c/users/5356618994a14b1ebab8f28f60a5a04d/roles/0e78444e43c744668178971af4087ca0` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:00.810683 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc8c8fd-3e29-434a-ac45-d7e1435fd62b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c32ba0a380a54b69a68a162a0194b45c, user_id=5356618994a14b1ebab8f28f60a5a04d, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:00.814509 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc8c8fd-3e29-434a-ac45-d7e1435fd62b admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:00.828091 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 920/1840] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:52:00 2022] PUT /identity/v3/projects/c32ba0a380a54b69a68a162a0194b45c/users/5356618994a14b1ebab8f28f60a5a04d/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:00.841061 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497a51d-740c-4a37-9f21-c476a196fa0e None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:00.841346 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497a51d-740c-4a37-9f21-c476a196fa0e None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:00.841617 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497a51d-740c-4a37-9f21-c476a196fa0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:00.884166 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-4497a51d-740c-4a37-9f21-c476a196fa0e None None] MFA Rules not processed for user `5356618994a14b1ebab8f28f60a5a04d`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:00.886926 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-4497a51d-740c-4a37-9f21-c476a196fa0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:00.968681 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 921/1841] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:00 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:00.980702 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc8c8fd-3e29-434a-ac45-d7e1435fd62b admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:00.982540 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-1cc8c8fd-3e29-434a-ac45-d7e1435fd62b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:01.041032 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:01.046812 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc8c8fd-3e29-434a-ac45-d7e1435fd62b admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:01.051663 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda8dc66-38a1-4b7b-8cf6-41fd497b5cf3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:01.052463 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8dc66-38a1-4b7b-8cf6-41fd497b5cf3 service cinder] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:01.053235 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8dc66-38a1-4b7b-8cf6-41fd497b5cf3 service cinder] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:01.055036 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8dc66-38a1-4b7b-8cf6-41fd497b5cf3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:01.072052 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda8dc66-38a1-4b7b-8cf6-41fd497b5cf3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:01.073736 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda8dc66-38a1-4b7b-8cf6-41fd497b5cf3 service cinder] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:01.087110 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 921/1842] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:52:00 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:52:01.510321 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4497a51d-740c-4a37-9f21-c476a196fa0e None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:01.582289 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:01.598779 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4497a51d-740c-4a37-9f21-c476a196fa0e None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:01.605434 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e52275-f42a-4ead-ae7f-9c34f6369a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:01.606224 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e52275-f42a-4ead-ae7f-9c34f6369a95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:01.606428 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e52275-f42a-4ead-ae7f-9c34f6369a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:01.607681 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e52275-f42a-4ead-ae7f-9c34f6369a95 admin admin] PATH_INFO: `/v3/users/0dbdc88d8afd423da8b86ff5bf51494e` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:01.608910 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e52275-f42a-4ead-ae7f-9c34f6369a95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dbdc88d8afd423da8b86ff5bf51494e)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:01.610809 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e52275-f42a-4ead-ae7f-9c34f6369a95 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:01.711556 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-e9e52275-f42a-4ead-ae7f-9c34f6369a95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dbdc88d8afd423da8b86ff5bf51494e'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:01.712672 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-e9e52275-f42a-4ead-ae7f-9c34f6369a95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dbdc88d8afd423da8b86ff5bf51494e'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:01.713517 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-e9e52275-f42a-4ead-ae7f-9c34f6369a95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dbdc88d8afd423da8b86ff5bf51494e'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:01.733277 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-e9e52275-f42a-4ead-ae7f-9c34f6369a95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dbdc88d8afd423da8b86ff5bf51494e'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:01.743229 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-e9e52275-f42a-4ead-ae7f-9c34f6369a95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dbdc88d8afd423da8b86ff5bf51494e'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:01.758910 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 922/1843] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:52:01 2022] DELETE /identity/v3/users/0dbdc88d8afd423da8b86ff5bf51494e => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:01.763872 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07515c7-8c29-486c-86f6-933b461bd2f8 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:01.764539 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07515c7-8c29-486c-86f6-933b461bd2f8 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:01.764833 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07515c7-8c29-486c-86f6-933b461bd2f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:01.800207 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-e07515c7-8c29-486c-86f6-933b461bd2f8 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:01.802472 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-e07515c7-8c29-486c-86f6-933b461bd2f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:01.918608 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 922/1844] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:01.931878 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e52275-f42a-4ead-ae7f-9c34f6369a95 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:01.933396 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-e9e52275-f42a-4ead-ae7f-9c34f6369a95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:02.032183 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e52275-f42a-4ead-ae7f-9c34f6369a95 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:02.037779 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d179f-8c19-4440-8024-f6160eb9dafc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:02.038708 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d179f-8c19-4440-8024-f6160eb9dafc service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:02.039418 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d179f-8c19-4440-8024-f6160eb9dafc service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:02.040002 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d179f-8c19-4440-8024-f6160eb9dafc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:02.051461 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7d179f-8c19-4440-8024-f6160eb9dafc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:02.053868 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7d179f-8c19-4440-8024-f6160eb9dafc service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:02.083040 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 923/1845] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:52:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:52:02.187841 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea47e7e-bbac-44ba-82b6-6dad4894c9f0 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:02.188354 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea47e7e-bbac-44ba-82b6-6dad4894c9f0 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:02.188604 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea47e7e-bbac-44ba-82b6-6dad4894c9f0 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:02.190348 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 923/1846] 10.222.0.20 () {54 vars in 921 bytes} [Tue Jul 26 12:52:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:52:02.200046 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a911ab-7548-411f-ad67-494c6f53f362 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:02.200394 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a911ab-7548-411f-ad67-494c6f53f362 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:02.200733 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a911ab-7548-411f-ad67-494c6f53f362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:02.235161 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-95a911ab-7548-411f-ad67-494c6f53f362 None None] MFA Rules not processed for user `0f5d6f3c12244c0faf26048513dc49e7`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:02.236739 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-95a911ab-7548-411f-ad67-494c6f53f362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:02.381258 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 924/1847] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Jul 26 12:52:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:02.395003 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea47e7e-bbac-44ba-82b6-6dad4894c9f0 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:02.397275 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-aea47e7e-bbac-44ba-82b6-6dad4894c9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:02.433855 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea47e7e-bbac-44ba-82b6-6dad4894c9f0 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:02.449712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3eca2e-fe49-481e-af06-40a4324b42b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:02.450380 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3eca2e-fe49-481e-af06-40a4324b42b9 service nova] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:02.450665 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3eca2e-fe49-481e-af06-40a4324b42b9 service nova] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:02.450935 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3eca2e-fe49-481e-af06-40a4324b42b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:02.464291 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3eca2e-fe49-481e-af06-40a4324b42b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:02.466104 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3eca2e-fe49-481e-af06-40a4324b42b9 service nova] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:02.478484 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 924/1848] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:52:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:52:02.801287 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a911ab-7548-411f-ad67-494c6f53f362 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:02.802713 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-95a911ab-7548-411f-ad67-494c6f53f362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:02.813791 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:02.822315 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a911ab-7548-411f-ad67-494c6f53f362 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:02.828475 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df83dbe8-5c67-4670-9d3b-e0d4c469e14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:02.829384 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83dbe8-5c67-4670-9d3b-e0d4c469e14f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:02.829729 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83dbe8-5c67-4670-9d3b-e0d4c469e14f admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:02.830084 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83dbe8-5c67-4670-9d3b-e0d4c469e14f admin admin] PATH_INFO: `/v3/projects/f8702a1ddd484978965e616131dff4f3` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:02.831389 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df83dbe8-5c67-4670-9d3b-e0d4c469e14f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8702a1ddd484978965e616131dff4f3)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:02.833442 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df83dbe8-5c67-4670-9d3b-e0d4c469e14f admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:02.878997 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3eca2e-fe49-481e-af06-40a4324b42b9 service nova] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:02.880614 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-be3eca2e-fe49-481e-af06-40a4324b42b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:02.898287 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3eca2e-fe49-481e-af06-40a4324b42b9 service nova] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:02.909202 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-df83dbe8-5c67-4670-9d3b-e0d4c469e14f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8702a1ddd484978965e616131dff4f3', 'request_id': 'req-df83dbe8-5c67-4670-9d3b-e0d4c469e14f'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:02.917304 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-df83dbe8-5c67-4670-9d3b-e0d4c469e14f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8702a1ddd484978965e616131dff4f3', 'request_id': 'req-df83dbe8-5c67-4670-9d3b-e0d4c469e14f'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:02.931288 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-df83dbe8-5c67-4670-9d3b-e0d4c469e14f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8702a1ddd484978965e616131dff4f3', 'request_id': 'req-df83dbe8-5c67-4670-9d3b-e0d4c469e14f'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:02.946218 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-df83dbe8-5c67-4670-9d3b-e0d4c469e14f admin admin] The token cache is being invalidate because project f8702a1ddd484978965e616131dff4f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:52:02.946708 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-df83dbe8-5c67-4670-9d3b-e0d4c469e14f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:02.948258 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-df83dbe8-5c67-4670-9d3b-e0d4c469e14f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:02.950529 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 925/1849] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:52:02 2022] DELETE /identity/v3/projects/f8702a1ddd484978965e616131dff4f3 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:02.955378 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df83dbe8-5c67-4670-9d3b-e0d4c469e14f admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:02.956831 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-df83dbe8-5c67-4670-9d3b-e0d4c469e14f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:02.991842 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebe273-ae4d-49b1-8b1e-f68f056e81a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:02.992712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebe273-ae4d-49b1-8b1e-f68f056e81a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:02.993166 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebe273-ae4d-49b1-8b1e-f68f056e81a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:02.993599 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebe273-ae4d-49b1-8b1e-f68f056e81a1 admin admin] PATH_INFO: `/v3/users/2333123d25ee44a48d9d924a7d9fbd12` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:02.995404 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ebe273-ae4d-49b1-8b1e-f68f056e81a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2333123d25ee44a48d9d924a7d9fbd12)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:02.998439 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ebe273-ae4d-49b1-8b1e-f68f056e81a1 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:03.065875 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df83dbe8-5c67-4670-9d3b-e0d4c469e14f admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:03.071297 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61640a2d-4770-4d24-9827-1da676b8c7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:03.071990 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61640a2d-4770-4d24-9827-1da676b8c7e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:03.072315 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61640a2d-4770-4d24-9827-1da676b8c7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:03.072593 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61640a2d-4770-4d24-9827-1da676b8c7e6 admin admin] PATH_INFO: `/v3/users/5356618994a14b1ebab8f28f60a5a04d` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:03.073852 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61640a2d-4770-4d24-9827-1da676b8c7e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5356618994a14b1ebab8f28f60a5a04d)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:03.076572 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61640a2d-4770-4d24-9827-1da676b8c7e6 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:03.101102 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-82ebe273-ae4d-49b1-8b1e-f68f056e81a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2333123d25ee44a48d9d924a7d9fbd12'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:03.116574 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-82ebe273-ae4d-49b1-8b1e-f68f056e81a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2333123d25ee44a48d9d924a7d9fbd12'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:03.117234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-82ebe273-ae4d-49b1-8b1e-f68f056e81a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2333123d25ee44a48d9d924a7d9fbd12'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:03.130419 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-82ebe273-ae4d-49b1-8b1e-f68f056e81a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2333123d25ee44a48d9d924a7d9fbd12'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:03.143868 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:03.147454 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-82ebe273-ae4d-49b1-8b1e-f68f056e81a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2333123d25ee44a48d9d924a7d9fbd12'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:03.161877 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 925/1850] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:52:02 2022] DELETE /identity/v3/users/2333123d25ee44a48d9d924a7d9fbd12 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:03.168788 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce691b7-20d7-480e-82fe-04551d976954 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:03.169082 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce691b7-20d7-480e-82fe-04551d976954 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:03.169323 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce691b7-20d7-480e-82fe-04551d976954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:03.174581 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-61640a2d-4770-4d24-9827-1da676b8c7e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5356618994a14b1ebab8f28f60a5a04d'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:03.175861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-61640a2d-4770-4d24-9827-1da676b8c7e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5356618994a14b1ebab8f28f60a5a04d'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:03.177033 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-61640a2d-4770-4d24-9827-1da676b8c7e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5356618994a14b1ebab8f28f60a5a04d'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:03.197858 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-61640a2d-4770-4d24-9827-1da676b8c7e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5356618994a14b1ebab8f28f60a5a04d'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:03.211548 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-61640a2d-4770-4d24-9827-1da676b8c7e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5356618994a14b1ebab8f28f60a5a04d'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:03.213272 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-7ce691b7-20d7-480e-82fe-04551d976954 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:03.214745 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-7ce691b7-20d7-480e-82fe-04551d976954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:03.236106 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 926/1851] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:52:02 2022] DELETE /identity/v3/users/5356618994a14b1ebab8f28f60a5a04d => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:03.335847 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 926/1852] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:03.361121 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61640a2d-4770-4d24-9827-1da676b8c7e6 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:03.362755 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-61640a2d-4770-4d24-9827-1da676b8c7e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:03.375723 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:03.435464 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61640a2d-4770-4d24-9827-1da676b8c7e6 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:03.439426 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5c8de2-e023-4036-a023-8ccfb063f9ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:03.440095 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c8de2-e023-4036-a023-8ccfb063f9ce service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:03.440377 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c8de2-e023-4036-a023-8ccfb063f9ce service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:03.440653 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c8de2-e023-4036-a023-8ccfb063f9ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:03.506038 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5c8de2-e023-4036-a023-8ccfb063f9ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:03.507508 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5c8de2-e023-4036-a023-8ccfb063f9ce service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:03.527126 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 927/1853] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:52:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:52:03.814428 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce691b7-20d7-480e-82fe-04551d976954 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:03.816321 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-7ce691b7-20d7-480e-82fe-04551d976954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:03.831322 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce691b7-20d7-480e-82fe-04551d976954 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:03.836962 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426eb439-4b26-4a4d-8a64-59ab44817117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:03.837757 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426eb439-4b26-4a4d-8a64-59ab44817117 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:03.838040 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426eb439-4b26-4a4d-8a64-59ab44817117 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:03.838316 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426eb439-4b26-4a4d-8a64-59ab44817117 admin admin] PATH_INFO: `/v3/projects/c32ba0a380a54b69a68a162a0194b45c` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:03.839807 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426eb439-4b26-4a4d-8a64-59ab44817117 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c32ba0a380a54b69a68a162a0194b45c)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:03.841968 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426eb439-4b26-4a4d-8a64-59ab44817117 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:03.873369 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5c8de2-e023-4036-a023-8ccfb063f9ce service neutron] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:03.874974 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-dd5c8de2-e023-4036-a023-8ccfb063f9ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:03.893364 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5c8de2-e023-4036-a023-8ccfb063f9ce service neutron] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:03.900863 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505daee7-cfd6-4f6a-a6d2-4bac9a005ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:03.901994 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505daee7-cfd6-4f6a-a6d2-4bac9a005ed7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:03.902495 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505daee7-cfd6-4f6a-a6d2-4bac9a005ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:03.902849 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505daee7-cfd6-4f6a-a6d2-4bac9a005ed7 admin admin] PATH_INFO: `/v3/projects/1cc07f98ade3464e91dc6ae904358e43` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:03.904234 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505daee7-cfd6-4f6a-a6d2-4bac9a005ed7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cc07f98ade3464e91dc6ae904358e43)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:03.906409 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505daee7-cfd6-4f6a-a6d2-4bac9a005ed7 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:03.924182 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-426eb439-4b26-4a4d-8a64-59ab44817117 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c32ba0a380a54b69a68a162a0194b45c', 'request_id': 'req-426eb439-4b26-4a4d-8a64-59ab44817117'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:03.931313 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-426eb439-4b26-4a4d-8a64-59ab44817117 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c32ba0a380a54b69a68a162a0194b45c', 'request_id': 'req-426eb439-4b26-4a4d-8a64-59ab44817117'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:03.945360 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-426eb439-4b26-4a4d-8a64-59ab44817117 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c32ba0a380a54b69a68a162a0194b45c', 'request_id': 'req-426eb439-4b26-4a4d-8a64-59ab44817117'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:03.968302 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-505daee7-cfd6-4f6a-a6d2-4bac9a005ed7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cc07f98ade3464e91dc6ae904358e43', 'request_id': 'req-505daee7-cfd6-4f6a-a6d2-4bac9a005ed7'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:03.970799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-426eb439-4b26-4a4d-8a64-59ab44817117 admin admin] The token cache is being invalidate because project c32ba0a380a54b69a68a162a0194b45c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:52:03.971093 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-426eb439-4b26-4a4d-8a64-59ab44817117 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:03.972460 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-426eb439-4b26-4a4d-8a64-59ab44817117 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:03.978131 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 927/1854] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:52:03 2022] DELETE /identity/v3/projects/c32ba0a380a54b69a68a162a0194b45c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:03.980043 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c897d56-e1d7-4741-93b1-9780d26aedba None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:03.980353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c897d56-e1d7-4741-93b1-9780d26aedba None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:03.980639 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c897d56-e1d7-4741-93b1-9780d26aedba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:03.983081 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-505daee7-cfd6-4f6a-a6d2-4bac9a005ed7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cc07f98ade3464e91dc6ae904358e43', 'request_id': 'req-505daee7-cfd6-4f6a-a6d2-4bac9a005ed7'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:03.994005 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-505daee7-cfd6-4f6a-a6d2-4bac9a005ed7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cc07f98ade3464e91dc6ae904358e43', 'request_id': 'req-505daee7-cfd6-4f6a-a6d2-4bac9a005ed7'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:04.011585 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-2c897d56-e1d7-4741-93b1-9780d26aedba None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:04.013642 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-2c897d56-e1d7-4741-93b1-9780d26aedba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:04.013792 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-505daee7-cfd6-4f6a-a6d2-4bac9a005ed7 admin admin] The token cache is being invalidate because project 1cc07f98ade3464e91dc6ae904358e43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:52:04.014120 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-505daee7-cfd6-4f6a-a6d2-4bac9a005ed7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:04.016150 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-505daee7-cfd6-4f6a-a6d2-4bac9a005ed7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:04.018766 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 928/1855] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:52:03 2022] DELETE /identity/v3/projects/1cc07f98ade3464e91dc6ae904358e43 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:52:04.023606 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505daee7-cfd6-4f6a-a6d2-4bac9a005ed7 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:04.025057 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-505daee7-cfd6-4f6a-a6d2-4bac9a005ed7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:04.099526 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:04.160088 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 928/1856] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:04.163912 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479292fe-4555-4ab7-b87b-7494ce195689 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:04.164377 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479292fe-4555-4ab7-b87b-7494ce195689 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:04.164669 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479292fe-4555-4ab7-b87b-7494ce195689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:04.165279 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505daee7-cfd6-4f6a-a6d2-4bac9a005ed7 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:04.187720 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276731f3-4943-457e-938f-1a7b909fc4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:04.188537 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276731f3-4943-457e-938f-1a7b909fc4bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:04.188812 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276731f3-4943-457e-938f-1a7b909fc4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:04.189091 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276731f3-4943-457e-938f-1a7b909fc4bc admin admin] PATH_INFO: `/v3/users/7c4a230a981c4e629b68c97d3f30830b` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:04.195789 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276731f3-4943-457e-938f-1a7b909fc4bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c4a230a981c4e629b68c97d3f30830b)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:04.204834 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276731f3-4943-457e-938f-1a7b909fc4bc admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:04.209345 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-479292fe-4555-4ab7-b87b-7494ce195689 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:04.211350 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-479292fe-4555-4ab7-b87b-7494ce195689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:04.234265 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 929/1857] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:52:04.238937 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479292fe-4555-4ab7-b87b-7494ce195689 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:04.259195 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479292fe-4555-4ab7-b87b-7494ce195689 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:04.272449 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7419830-14f7-4d67-bad9-43ff0bb735e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:04.273357 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7419830-14f7-4d67-bad9-43ff0bb735e2 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:04.273684 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7419830-14f7-4d67-bad9-43ff0bb735e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:04.274797 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7419830-14f7-4d67-bad9-43ff0bb735e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:04.275328 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7419830-14f7-4d67-bad9-43ff0bb735e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:52:04.275758 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7419830-14f7-4d67-bad9-43ff0bb735e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:04.282096 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7419830-14f7-4d67-bad9-43ff0bb735e2 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:04.309525 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:04.326980 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 930/1858] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:52:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:52:04.327769 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-276731f3-4943-457e-938f-1a7b909fc4bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c4a230a981c4e629b68c97d3f30830b'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:04.330639 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7419830-14f7-4d67-bad9-43ff0bb735e2 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:04.330778 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-276731f3-4943-457e-938f-1a7b909fc4bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c4a230a981c4e629b68c97d3f30830b'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:04.336558 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-276731f3-4943-457e-938f-1a7b909fc4bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c4a230a981c4e629b68c97d3f30830b'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:04.355045 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7419830-14f7-4d67-bad9-43ff0bb735e2 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:04.361827 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-276731f3-4943-457e-938f-1a7b909fc4bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c4a230a981c4e629b68c97d3f30830b'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:04.369359 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ef2da-1f10-4c0d-a0bd-adcf37670a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:04.370101 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ef2da-1f10-4c0d-a0bd-adcf37670a6a admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:04.370484 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ef2da-1f10-4c0d-a0bd-adcf37670a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:04.370829 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ef2da-1f10-4c0d-a0bd-adcf37670a6a admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:04.371772 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1ef2da-1f10-4c0d-a0bd-adcf37670a6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:04.374062 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1ef2da-1f10-4c0d-a0bd-adcf37670a6a admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:04.385895 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-276731f3-4943-457e-938f-1a7b909fc4bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c4a230a981c4e629b68c97d3f30830b'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:04.417800 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 931/1860] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:52:04 2022] POST /identity/v3/projects => generated 383 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:52:04.423484 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 929/1860] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:52:04 2022] DELETE /identity/v3/users/7c4a230a981c4e629b68c97d3f30830b => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:04.424647 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ef2da-1f10-4c0d-a0bd-adcf37670a6a admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:04.427151 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-9a1ef2da-1f10-4c0d-a0bd-adcf37670a6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:04.524060 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ef2da-1f10-4c0d-a0bd-adcf37670a6a admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:04.530406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ebf84-dff4-42b1-bffe-e758d3e63b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:04.531841 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ebf84-dff4-42b1-bffe-e758d3e63b6a admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:04.532190 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ebf84-dff4-42b1-bffe-e758d3e63b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:04.532903 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ebf84-dff4-42b1-bffe-e758d3e63b6a admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:04.533353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79ebf84-dff4-42b1-bffe-e758d3e63b6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:04.535441 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79ebf84-dff4-42b1-bffe-e758d3e63b6a admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:04.573419 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 932/1861] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:52:04 2022] POST /identity/v3/users => generated 292 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:52:04.577898 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276731f3-4943-457e-938f-1a7b909fc4bc admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:04.724414 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276731f3-4943-457e-938f-1a7b909fc4bc admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:04.731047 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1943128e-406b-4326-b97d-18a8265de20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:04.732548 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1943128e-406b-4326-b97d-18a8265de20f admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:04.733156 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1943128e-406b-4326-b97d-18a8265de20f admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:04.734648 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1943128e-406b-4326-b97d-18a8265de20f admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:04.735988 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1943128e-406b-4326-b97d-18a8265de20f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:04.737727 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1943128e-406b-4326-b97d-18a8265de20f admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:04.753408 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 930/1862] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:04 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:52:04.757572 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ebf84-dff4-42b1-bffe-e758d3e63b6a admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:04.771424 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ebf84-dff4-42b1-bffe-e758d3e63b6a admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:04.781623 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45c3539-e4b5-48bb-a09b-0cce53579b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:04.783350 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c3539-e4b5-48bb-a09b-0cce53579b6a admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:04.783941 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c3539-e4b5-48bb-a09b-0cce53579b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:04.784572 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c3539-e4b5-48bb-a09b-0cce53579b6a admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:04.788582 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45c3539-e4b5-48bb-a09b-0cce53579b6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:04.791041 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45c3539-e4b5-48bb-a09b-0cce53579b6a admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:04.816628 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 933/1863] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:04 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:52:04.820744 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1943128e-406b-4326-b97d-18a8265de20f admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:04.844512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1943128e-406b-4326-b97d-18a8265de20f admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:04.853694 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8671b58d-c04c-4bc9-92dd-d8fe8263b451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:04.854685 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671b58d-c04c-4bc9-92dd-d8fe8263b451 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:04.855471 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671b58d-c04c-4bc9-92dd-d8fe8263b451 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:04.856105 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671b58d-c04c-4bc9-92dd-d8fe8263b451 admin admin] PATH_INFO: `/v3/projects/e79476d97a854574a7de69c947ffe470/users/ff646cd99262476fa5a06f490dc4107f/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:04.877203 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8671b58d-c04c-4bc9-92dd-d8fe8263b451 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e79476d97a854574a7de69c947ffe470, user_id=ff646cd99262476fa5a06f490dc4107f, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:04.882252 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8671b58d-c04c-4bc9-92dd-d8fe8263b451 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:04.910996 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 931/1864] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:52:04 2022] PUT /identity/v3/projects/e79476d97a854574a7de69c947ffe470/users/ff646cd99262476fa5a06f490dc4107f/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:52:04.923116 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b399ef95-c9e5-4638-abb4-dc4f990da12d None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:04.923429 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b399ef95-c9e5-4638-abb4-dc4f990da12d None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:04.923701 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b399ef95-c9e5-4638-abb4-dc4f990da12d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:04.974553 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8671b58d-c04c-4bc9-92dd-d8fe8263b451 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:04.977424 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-8671b58d-c04c-4bc9-92dd-d8fe8263b451 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:04.977685 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-b399ef95-c9e5-4638-abb4-dc4f990da12d None None] MFA Rules not processed for user `ff646cd99262476fa5a06f490dc4107f`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:04.978997 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-b399ef95-c9e5-4638-abb4-dc4f990da12d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:05.054635 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:05.117529 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8671b58d-c04c-4bc9-92dd-d8fe8263b451 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:05.124202 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 934/1865] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:05.125747 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b399ef95-c9e5-4638-abb4-dc4f990da12d None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:05.128321 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff5b9a-6ca8-4f4a-abe9-8f00114080a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:05.129376 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff5b9a-6ca8-4f4a-abe9-8f00114080a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:05.129858 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff5b9a-6ca8-4f4a-abe9-8f00114080a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:05.131572 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff5b9a-6ca8-4f4a-abe9-8f00114080a0 admin admin] PATH_INFO: `/v3/users/10678e4317e740a7ba3994e2fd993515` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:05.133264 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ff5b9a-6ca8-4f4a-abe9-8f00114080a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10678e4317e740a7ba3994e2fd993515)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:05.136684 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ff5b9a-6ca8-4f4a-abe9-8f00114080a0 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:05.138208 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b399ef95-c9e5-4638-abb4-dc4f990da12d None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:05.143511 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc4b24-5f55-46dc-910a-b9076c2fd508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:05.144472 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc4b24-5f55-46dc-910a-b9076c2fd508 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:05.144756 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc4b24-5f55-46dc-910a-b9076c2fd508 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:05.145029 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc4b24-5f55-46dc-910a-b9076c2fd508 admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:05.145841 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc4b24-5f55-46dc-910a-b9076c2fd508 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:05.148511 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc4b24-5f55-46dc-910a-b9076c2fd508 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:05.200010 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 935/1866] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:52:05 2022] POST /identity/v3/projects => generated 385 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:52:05.200010 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc4b24-5f55-46dc-910a-b9076c2fd508 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:05.204981 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-ebdc4b24-5f55-46dc-910a-b9076c2fd508 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:05.225834 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-37ff5b9a-6ca8-4f4a-abe9-8f00114080a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10678e4317e740a7ba3994e2fd993515'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:05.226542 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-37ff5b9a-6ca8-4f4a-abe9-8f00114080a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10678e4317e740a7ba3994e2fd993515'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:05.237338 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:05.240495 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-37ff5b9a-6ca8-4f4a-abe9-8f00114080a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10678e4317e740a7ba3994e2fd993515'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:05.269024 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-37ff5b9a-6ca8-4f4a-abe9-8f00114080a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10678e4317e740a7ba3994e2fd993515'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:05.282145 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc4b24-5f55-46dc-910a-b9076c2fd508 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:05.288746 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-37ff5b9a-6ca8-4f4a-abe9-8f00114080a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10678e4317e740a7ba3994e2fd993515'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:05.289680 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e52103-57fb-4712-8bb3-ed6d84aedd0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:05.290349 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e52103-57fb-4712-8bb3-ed6d84aedd0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:05.290628 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e52103-57fb-4712-8bb3-ed6d84aedd0f admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:05.291064 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e52103-57fb-4712-8bb3-ed6d84aedd0f admin admin] PATH_INFO: `/v3/projects/8ba249f0d79f45dab8d43bcf6dc71868` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:05.292350 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e52103-57fb-4712-8bb3-ed6d84aedd0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ba249f0d79f45dab8d43bcf6dc71868)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:05.295699 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e52103-57fb-4712-8bb3-ed6d84aedd0f admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:05.305970 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 932/1867] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:52:04 2022] DELETE /identity/v3/users/10678e4317e740a7ba3994e2fd993515 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:05.306972 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff5b9a-6ca8-4f4a-abe9-8f00114080a0 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:05.308431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-37ff5b9a-6ca8-4f4a-abe9-8f00114080a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:05.358920 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-94e52103-57fb-4712-8bb3-ed6d84aedd0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ba249f0d79f45dab8d43bcf6dc71868', 'request_id': 'req-94e52103-57fb-4712-8bb3-ed6d84aedd0f'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:05.367900 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-94e52103-57fb-4712-8bb3-ed6d84aedd0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ba249f0d79f45dab8d43bcf6dc71868', 'request_id': 'req-94e52103-57fb-4712-8bb3-ed6d84aedd0f'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:05.379300 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-94e52103-57fb-4712-8bb3-ed6d84aedd0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ba249f0d79f45dab8d43bcf6dc71868', 'request_id': 'req-94e52103-57fb-4712-8bb3-ed6d84aedd0f'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:05.391986 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-94e52103-57fb-4712-8bb3-ed6d84aedd0f admin admin] The token cache is being invalidate because project 8ba249f0d79f45dab8d43bcf6dc71868 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:52:05.392330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-94e52103-57fb-4712-8bb3-ed6d84aedd0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:05.393065 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-94e52103-57fb-4712-8bb3-ed6d84aedd0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:05.395153 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 936/1868] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:52:05 2022] DELETE /identity/v3/projects/8ba249f0d79f45dab8d43bcf6dc71868 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:05.396188 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130cb01f-465e-4e8f-941a-e0e3e4cd4407 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:05.396989 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130cb01f-465e-4e8f-941a-e0e3e4cd4407 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:05.397271 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130cb01f-465e-4e8f-941a-e0e3e4cd4407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:05.402427 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff5b9a-6ca8-4f4a-abe9-8f00114080a0 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:05.424100 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-130cb01f-465e-4e8f-941a-e0e3e4cd4407 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:05.425487 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-130cb01f-465e-4e8f-941a-e0e3e4cd4407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:05.465991 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:05.474017 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66213e33-da86-4ffb-a9ca-65528a73332d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:05.475969 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66213e33-da86-4ffb-a9ca-65528a73332d admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:05.476787 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66213e33-da86-4ffb-a9ca-65528a73332d admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:05.478152 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66213e33-da86-4ffb-a9ca-65528a73332d admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:05.479230 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66213e33-da86-4ffb-a9ca-65528a73332d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:05.482320 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66213e33-da86-4ffb-a9ca-65528a73332d admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:05.498530 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 937/1869] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:05.501177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b4dd7-eea1-4139-9a18-f082393ef611 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:05.501410 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b4dd7-eea1-4139-9a18-f082393ef611 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:05.501832 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b4dd7-eea1-4139-9a18-f082393ef611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:05.512721 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 933/1870] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:52:05 2022] POST /identity/v3/users => generated 299 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:52:05.512721 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66213e33-da86-4ffb-a9ca-65528a73332d admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:05.515276 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-66213e33-da86-4ffb-a9ca-65528a73332d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:05.525402 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-363b4dd7-eea1-4139-9a18-f082393ef611 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:05.526825 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-363b4dd7-eea1-4139-9a18-f082393ef611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:05.535188 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 938/1871] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:05.543445 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363b4dd7-eea1-4139-9a18-f082393ef611 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:05.544718 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-363b4dd7-eea1-4139-9a18-f082393ef611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:05.563416 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363b4dd7-eea1-4139-9a18-f082393ef611 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:05.569054 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68db32b3-cdaa-4673-a1a3-2a3baba784bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:05.569696 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db32b3-cdaa-4673-a1a3-2a3baba784bf admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:05.569972 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db32b3-cdaa-4673-a1a3-2a3baba784bf admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:05.570241 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db32b3-cdaa-4673-a1a3-2a3baba784bf admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:05.571201 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68db32b3-cdaa-4673-a1a3-2a3baba784bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:05.573530 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68db32b3-cdaa-4673-a1a3-2a3baba784bf admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:05.587722 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 939/1872] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:05 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:52:05.588751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7ed56-465a-44f1-935c-c06fa6a9573e None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:05.589030 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7ed56-465a-44f1-935c-c06fa6a9573e None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:05.589308 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7ed56-465a-44f1-935c-c06fa6a9573e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:05.629896 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-c3c7ed56-465a-44f1-935c-c06fa6a9573e None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:05.631828 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-c3c7ed56-465a-44f1-935c-c06fa6a9573e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:05.640970 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66213e33-da86-4ffb-a9ca-65528a73332d admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:05.661012 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0edd676-cc1e-4628-9989-cfc38e5ec94f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:05.661584 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0edd676-cc1e-4628-9989-cfc38e5ec94f service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:05.661875 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0edd676-cc1e-4628-9989-cfc38e5ec94f service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:05.665710 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0edd676-cc1e-4628-9989-cfc38e5ec94f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:05.665878 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 940/1873] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:05.665878 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7ed56-465a-44f1-935c-c06fa6a9573e None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:05.684512 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7ed56-465a-44f1-935c-c06fa6a9573e None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:05.689885 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0edd676-cc1e-4628-9989-cfc38e5ec94f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:05.697538 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0edd676-cc1e-4628-9989-cfc38e5ec94f service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:05.702126 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b049ccd4-6c8b-47d0-b58d-ca81e6f3e9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:05.702820 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b049ccd4-6c8b-47d0-b58d-ca81e6f3e9f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:05.703109 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b049ccd4-6c8b-47d0-b58d-ca81e6f3e9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:05.703380 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b049ccd4-6c8b-47d0-b58d-ca81e6f3e9f6 admin admin] PATH_INFO: `/v3/projects/a227bb3ec80e47dbb1a11e4bf00251e9/users/2dc8b56998f944398ea495bd1b276ad9/roles/0e78444e43c744668178971af4087ca0` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:05.727182 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b049ccd4-6c8b-47d0-b58d-ca81e6f3e9f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a227bb3ec80e47dbb1a11e4bf00251e9, user_id=2dc8b56998f944398ea495bd1b276ad9, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:05.727385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 934/1874] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:52:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:52:05.728168 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0edd676-cc1e-4628-9989-cfc38e5ec94f service neutron] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:05.730072 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b049ccd4-6c8b-47d0-b58d-ca81e6f3e9f6 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:05.740572 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0edd676-cc1e-4628-9989-cfc38e5ec94f service neutron] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:05.753205 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a39809f-05bf-4d44-97c6-cf5614952740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:05.754738 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a39809f-05bf-4d44-97c6-cf5614952740 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:05.755068 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a39809f-05bf-4d44-97c6-cf5614952740 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:05.755340 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a39809f-05bf-4d44-97c6-cf5614952740 admin admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:05.757092 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a39809f-05bf-4d44-97c6-cf5614952740 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:52:05.757516 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a39809f-05bf-4d44-97c6-cf5614952740 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:05.761729 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a39809f-05bf-4d44-97c6-cf5614952740 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:05.767254 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 941/1875] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:52:05 2022] PUT /identity/v3/projects/a227bb3ec80e47dbb1a11e4bf00251e9/users/2dc8b56998f944398ea495bd1b276ad9/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:05.772702 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b049ccd4-6c8b-47d0-b58d-ca81e6f3e9f6 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:05.780073 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 935/1876] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:52:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:52:05.784237 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a39809f-05bf-4d44-97c6-cf5614952740 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:05.790225 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:05.808628 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a39809f-05bf-4d44-97c6-cf5614952740 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:05.881874 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b049ccd4-6c8b-47d0-b58d-ca81e6f3e9f6 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:05.891349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697c2510-7eb9-4f86-9419-de3ddab3a720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:05.892092 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c2510-7eb9-4f86-9419-de3ddab3a720 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:05.893202 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c2510-7eb9-4f86-9419-de3ddab3a720 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:05.893488 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c2510-7eb9-4f86-9419-de3ddab3a720 admin admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:05.894189 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697c2510-7eb9-4f86-9419-de3ddab3a720 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:05.895764 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d965f0b1-b4c6-4e6b-8137-db29c1b81c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:05.896399 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965f0b1-b4c6-4e6b-8137-db29c1b81c47 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:05.896774 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965f0b1-b4c6-4e6b-8137-db29c1b81c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:05.897120 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965f0b1-b4c6-4e6b-8137-db29c1b81c47 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:05.897980 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d965f0b1-b4c6-4e6b-8137-db29c1b81c47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:05.901564 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697c2510-7eb9-4f86-9419-de3ddab3a720 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:05.902992 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d965f0b1-b4c6-4e6b-8137-db29c1b81c47 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:05.933816 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 942/1877] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:05 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:52:05.941739 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d965f0b1-b4c6-4e6b-8137-db29c1b81c47 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:05.958617 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d965f0b1-b4c6-4e6b-8137-db29c1b81c47 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:05.977109 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45abebd2-fcf1-4146-9618-db1d1fd78436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:05.977808 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45abebd2-fcf1-4146-9618-db1d1fd78436 admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:05.978081 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45abebd2-fcf1-4146-9618-db1d1fd78436 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:05.978347 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45abebd2-fcf1-4146-9618-db1d1fd78436 admin admin] PATH_INFO: `/v3/projects/a227bb3ec80e47dbb1a11e4bf00251e9/users/2dc8b56998f944398ea495bd1b276ad9/roles/0e78444e43c744668178971af4087ca0` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:05.979905 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 936/1878] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:52:05 2022] POST /identity/v3/projects => generated 383 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:52:05.982080 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45abebd2-fcf1-4146-9618-db1d1fd78436 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a227bb3ec80e47dbb1a11e4bf00251e9, user_id=2dc8b56998f944398ea495bd1b276ad9, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:05.986427 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697c2510-7eb9-4f86-9419-de3ddab3a720 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:05.987056 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45abebd2-fcf1-4146-9618-db1d1fd78436 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:06.014998 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 943/1879] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:52:05 2022] PUT /identity/v3/projects/a227bb3ec80e47dbb1a11e4bf00251e9/users/2dc8b56998f944398ea495bd1b276ad9/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:06.029536 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3a52f-b0eb-4d4d-bb23-80f9e7a4927d None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:06.029825 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3a52f-b0eb-4d4d-bb23-80f9e7a4927d None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:06.031956 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3a52f-b0eb-4d4d-bb23-80f9e7a4927d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:06.078197 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-d2a3a52f-b0eb-4d4d-bb23-80f9e7a4927d None None] MFA Rules not processed for user `2dc8b56998f944398ea495bd1b276ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:06.079557 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-d2a3a52f-b0eb-4d4d-bb23-80f9e7a4927d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:06.156136 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697c2510-7eb9-4f86-9419-de3ddab3a720 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:06.191035 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:06.205475 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 944/1880] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:06 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:06.215777 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3a52f-b0eb-4d4d-bb23-80f9e7a4927d None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:06.217740 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-d2a3a52f-b0eb-4d4d-bb23-80f9e7a4927d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:06.282823 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e0362-cb36-4e72-bba5-2fd14a402727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:06.283876 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e0362-cb36-4e72-bba5-2fd14a402727 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:06.285435 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e0362-cb36-4e72-bba5-2fd14a402727 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:06.287623 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e0362-cb36-4e72-bba5-2fd14a402727 admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:06.288464 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e0362-cb36-4e72-bba5-2fd14a402727 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:06.294152 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e0362-cb36-4e72-bba5-2fd14a402727 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:06.356018 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 937/1881] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:52:05 2022] POST /identity/v3/users => generated 292 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:52:06.361410 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e0362-cb36-4e72-bba5-2fd14a402727 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:06.374379 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3a52f-b0eb-4d4d-bb23-80f9e7a4927d None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:06.398225 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e0362-cb36-4e72-bba5-2fd14a402727 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:06.403682 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a107ab4-09a3-4ac8-a5df-324f443c922f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:06.404846 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a107ab4-09a3-4ac8-a5df-324f443c922f admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:06.405185 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a107ab4-09a3-4ac8-a5df-324f443c922f admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:06.405462 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a107ab4-09a3-4ac8-a5df-324f443c922f admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:06.406293 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a107ab4-09a3-4ac8-a5df-324f443c922f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:06.407971 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a107ab4-09a3-4ac8-a5df-324f443c922f admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:06.413684 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227ce97c-e8a1-40c1-8de2-3d1cb5bfa672 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:06.414333 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227ce97c-e8a1-40c1-8de2-3d1cb5bfa672 service cinder] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:06.414618 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227ce97c-e8a1-40c1-8de2-3d1cb5bfa672 service cinder] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:06.418354 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227ce97c-e8a1-40c1-8de2-3d1cb5bfa672 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:06.420797 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-227ce97c-e8a1-40c1-8de2-3d1cb5bfa672 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:06.432453 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 938/1882] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:52:06.433370 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a107ab4-09a3-4ac8-a5df-324f443c922f admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:06.437772 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-4a107ab4-09a3-4ac8-a5df-324f443c922f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:06.463344 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a107ab4-09a3-4ac8-a5df-324f443c922f admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:06.476976 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6739f3-72a7-40c8-b7e2-28b6d89fada2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:06.477633 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6739f3-72a7-40c8-b7e2-28b6d89fada2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:06.477915 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6739f3-72a7-40c8-b7e2-28b6d89fada2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:06.478202 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6739f3-72a7-40c8-b7e2-28b6d89fada2 admin admin] PATH_INFO: `/v3/projects/c95ed175d48e45ec9755854bf44948d5` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:06.479423 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6739f3-72a7-40c8-b7e2-28b6d89fada2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c95ed175d48e45ec9755854bf44948d5)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:06.481467 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6739f3-72a7-40c8-b7e2-28b6d89fada2 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:06.494322 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:06.548408 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227ce97c-e8a1-40c1-8de2-3d1cb5bfa672 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:06.550046 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227ce97c-e8a1-40c1-8de2-3d1cb5bfa672 service cinder] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:06.569492 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 945/1883] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:52:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 12:52:06.569492 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227ce97c-e8a1-40c1-8de2-3d1cb5bfa672 service cinder] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:06.581256 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-3a6739f3-72a7-40c8-b7e2-28b6d89fada2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c95ed175d48e45ec9755854bf44948d5', 'request_id': 'req-3a6739f3-72a7-40c8-b7e2-28b6d89fada2'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:06.591364 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-3a6739f3-72a7-40c8-b7e2-28b6d89fada2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c95ed175d48e45ec9755854bf44948d5', 'request_id': 'req-3a6739f3-72a7-40c8-b7e2-28b6d89fada2'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:06.603285 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227ce97c-e8a1-40c1-8de2-3d1cb5bfa672 service cinder] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:06.603846 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-3a6739f3-72a7-40c8-b7e2-28b6d89fada2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c95ed175d48e45ec9755854bf44948d5', 'request_id': 'req-3a6739f3-72a7-40c8-b7e2-28b6d89fada2'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:06.628579 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-3a6739f3-72a7-40c8-b7e2-28b6d89fada2 admin admin] The token cache is being invalidate because project c95ed175d48e45ec9755854bf44948d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:52:06.629436 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-3a6739f3-72a7-40c8-b7e2-28b6d89fada2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:06.631767 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-3a6739f3-72a7-40c8-b7e2-28b6d89fada2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:06.633961 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 939/1884] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:52:06 2022] DELETE /identity/v3/projects/c95ed175d48e45ec9755854bf44948d5 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:06.637657 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6739f3-72a7-40c8-b7e2-28b6d89fada2 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:06.639383 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-3a6739f3-72a7-40c8-b7e2-28b6d89fada2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:06.696418 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d30b76-2c8e-423a-ae47-6bd9485da33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:06.696847 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d30b76-2c8e-423a-ae47-6bd9485da33f admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:06.697133 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d30b76-2c8e-423a-ae47-6bd9485da33f admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:06.697534 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d30b76-2c8e-423a-ae47-6bd9485da33f admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:06.698439 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d30b76-2c8e-423a-ae47-6bd9485da33f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:06.699886 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d30b76-2c8e-423a-ae47-6bd9485da33f admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:06.712883 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 946/1885] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:52:06.717753 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d30b76-2c8e-423a-ae47-6bd9485da33f admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:06.720131 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-e3d30b76-2c8e-423a-ae47-6bd9485da33f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:06.720687 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6739f3-72a7-40c8-b7e2-28b6d89fada2 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:06.727093 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d545bb-3ec0-4fb5-9a2c-3e8a390c1f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:06.728720 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d545bb-3ec0-4fb5-9a2c-3e8a390c1f68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:06.729093 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d545bb-3ec0-4fb5-9a2c-3e8a390c1f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:06.729462 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d545bb-3ec0-4fb5-9a2c-3e8a390c1f68 admin admin] PATH_INFO: `/v3/users/4d3b5381f14f4f1d93b4fc0cfefc6b47` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:06.731111 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d545bb-3ec0-4fb5-9a2c-3e8a390c1f68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d3b5381f14f4f1d93b4fc0cfefc6b47)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:06.734477 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d545bb-3ec0-4fb5-9a2c-3e8a390c1f68 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:06.736654 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d30b76-2c8e-423a-ae47-6bd9485da33f admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:06.749615 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:06.763852 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c35d2f-e63c-4032-b3c6-b47d89301c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:06.763852 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c35d2f-e63c-4032-b3c6-b47d89301c29 admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:06.763852 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c35d2f-e63c-4032-b3c6-b47d89301c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:06.763852 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c35d2f-e63c-4032-b3c6-b47d89301c29 admin admin] PATH_INFO: `/v3/projects/c82f5e83a6804126bcef77e595ed076f/users/fbffc2026f394be2a1b9c54ce9b96f79/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:06.778751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c35d2f-e63c-4032-b3c6-b47d89301c29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c82f5e83a6804126bcef77e595ed076f, user_id=fbffc2026f394be2a1b9c54ce9b96f79, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:06.781208 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c35d2f-e63c-4032-b3c6-b47d89301c29 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:06.881180 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 947/1886] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:52:06 2022] PUT /identity/v3/projects/c82f5e83a6804126bcef77e595ed076f/users/fbffc2026f394be2a1b9c54ce9b96f79/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:06.893737 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2612e5-147f-4b34-913c-238aa63f5c1f None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:06.894038 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2612e5-147f-4b34-913c-238aa63f5c1f None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:06.894449 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2612e5-147f-4b34-913c-238aa63f5c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:06.921498 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-51d545bb-3ec0-4fb5-9a2c-3e8a390c1f68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d3b5381f14f4f1d93b4fc0cfefc6b47'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:06.923722 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-51d545bb-3ec0-4fb5-9a2c-3e8a390c1f68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d3b5381f14f4f1d93b4fc0cfefc6b47'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:06.925405 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-51d545bb-3ec0-4fb5-9a2c-3e8a390c1f68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d3b5381f14f4f1d93b4fc0cfefc6b47'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:06.948111 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-51d545bb-3ec0-4fb5-9a2c-3e8a390c1f68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d3b5381f14f4f1d93b4fc0cfefc6b47'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:06.956970 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-51d545bb-3ec0-4fb5-9a2c-3e8a390c1f68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d3b5381f14f4f1d93b4fc0cfefc6b47'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:06.979417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-af2612e5-147f-4b34-913c-238aa63f5c1f None None] MFA Rules not processed for user `fbffc2026f394be2a1b9c54ce9b96f79`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:06.982449 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 940/1887] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:52:06 2022] DELETE /identity/v3/users/4d3b5381f14f4f1d93b4fc0cfefc6b47 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:52:06.983418 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-af2612e5-147f-4b34-913c-238aa63f5c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:07.204690 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 948/1888] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:52:07.206172 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d545bb-3ec0-4fb5-9a2c-3e8a390c1f68 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:07.208446 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-51d545bb-3ec0-4fb5-9a2c-3e8a390c1f68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:07.368806 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d545bb-3ec0-4fb5-9a2c-3e8a390c1f68 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:07.374903 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ffa037-8b95-42a5-baf6-75ba0101cc72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:07.375737 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ffa037-8b95-42a5-baf6-75ba0101cc72 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:07.376732 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ffa037-8b95-42a5-baf6-75ba0101cc72 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:07.377722 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ffa037-8b95-42a5-baf6-75ba0101cc72 admin admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:07.379051 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ffa037-8b95-42a5-baf6-75ba0101cc72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:07.381352 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ffa037-8b95-42a5-baf6-75ba0101cc72 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:07.426643 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 941/1889] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:52:07 2022] POST /identity/v3/projects => generated 387 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:52:07.433096 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2612e5-147f-4b34-913c-238aa63f5c1f None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:07.453116 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2612e5-147f-4b34-913c-238aa63f5c1f None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:07.494188 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:07.527310 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43771f-1aa2-420f-a211-1c98d8aa7ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:07.527979 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43771f-1aa2-420f-a211-1c98d8aa7ebb admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:07.528262 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43771f-1aa2-420f-a211-1c98d8aa7ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:07.528536 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43771f-1aa2-420f-a211-1c98d8aa7ebb admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:07.529421 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf43771f-1aa2-420f-a211-1c98d8aa7ebb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:07.531419 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf43771f-1aa2-420f-a211-1c98d8aa7ebb admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:07.555386 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 949/1890] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:52:07 2022] POST /identity/v3/users => generated 300 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:52:07.559362 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ffa037-8b95-42a5-baf6-75ba0101cc72 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:07.572420 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ffa037-8b95-42a5-baf6-75ba0101cc72 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:07.577495 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3764c0f7-7627-4f04-9d32-f3e0a8434465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:07.578130 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3764c0f7-7627-4f04-9d32-f3e0a8434465 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:07.578420 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3764c0f7-7627-4f04-9d32-f3e0a8434465 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:07.578693 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3764c0f7-7627-4f04-9d32-f3e0a8434465 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:07.579422 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3764c0f7-7627-4f04-9d32-f3e0a8434465 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:07.581138 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3764c0f7-7627-4f04-9d32-f3e0a8434465 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:07.593232 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 942/1891] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:52:07.597637 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43771f-1aa2-420f-a211-1c98d8aa7ebb admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:07.608685 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43771f-1aa2-420f-a211-1c98d8aa7ebb admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:07.613493 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61ce06b-c4b0-4a90-8481-22728dbf62d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:07.614232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ce06b-c4b0-4a90-8481-22728dbf62d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:07.614440 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ce06b-c4b0-4a90-8481-22728dbf62d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:07.614717 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ce06b-c4b0-4a90-8481-22728dbf62d6 admin admin] PATH_INFO: `/v3/projects/0593ebe7f9104536b9723b63df5f08ac/users/771b1daa27f54605a1f8628050738e9e/roles/0e78444e43c744668178971af4087ca0` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:07.636062 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61ce06b-c4b0-4a90-8481-22728dbf62d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0593ebe7f9104536b9723b63df5f08ac, user_id=771b1daa27f54605a1f8628050738e9e, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:07.638873 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61ce06b-c4b0-4a90-8481-22728dbf62d6 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:07.657779 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 950/1892] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:52:07 2022] PUT /identity/v3/projects/0593ebe7f9104536b9723b63df5f08ac/users/771b1daa27f54605a1f8628050738e9e/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:07.659309 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3764c0f7-7627-4f04-9d32-f3e0a8434465 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:07.670755 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3764c0f7-7627-4f04-9d32-f3e0a8434465 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:07.722108 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61ce06b-c4b0-4a90-8481-22728dbf62d6 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:07.728016 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-e61ce06b-c4b0-4a90-8481-22728dbf62d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:07.780169 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b96e292-8b76-4bb3-9898-76c787e42946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:07.781521 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96e292-8b76-4bb3-9898-76c787e42946 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:07.781877 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96e292-8b76-4bb3-9898-76c787e42946 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:07.782362 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96e292-8b76-4bb3-9898-76c787e42946 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:07.783253 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b96e292-8b76-4bb3-9898-76c787e42946 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:07.785058 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b96e292-8b76-4bb3-9898-76c787e42946 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:07.795943 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:07.811596 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 943/1893] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:07 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:52:07.814644 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b96e292-8b76-4bb3-9898-76c787e42946 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:07.836650 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b96e292-8b76-4bb3-9898-76c787e42946 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:07.841542 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904286b4-5a7d-4787-91d5-0f3c645c8368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:07.842205 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904286b4-5a7d-4787-91d5-0f3c645c8368 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:07.842488 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904286b4-5a7d-4787-91d5-0f3c645c8368 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:07.842909 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904286b4-5a7d-4787-91d5-0f3c645c8368 admin admin] PATH_INFO: `/v3/projects/0593ebe7f9104536b9723b63df5f08ac/users/771b1daa27f54605a1f8628050738e9e/roles/0e78444e43c744668178971af4087ca0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:07.845760 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904286b4-5a7d-4787-91d5-0f3c645c8368 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0593ebe7f9104536b9723b63df5f08ac, user_id=771b1daa27f54605a1f8628050738e9e, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:07.848487 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904286b4-5a7d-4787-91d5-0f3c645c8368 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:07.863391 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 944/1894] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:52:07 2022] PUT /identity/v3/projects/0593ebe7f9104536b9723b63df5f08ac/users/771b1daa27f54605a1f8628050738e9e/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:07.874409 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f473c-5f78-4dcc-a609-332b139e7327 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:07.874753 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f473c-5f78-4dcc-a609-332b139e7327 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:07.874992 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f473c-5f78-4dcc-a609-332b139e7327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:07.884316 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61ce06b-c4b0-4a90-8481-22728dbf62d6 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:07.906371 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-a42f473c-5f78-4dcc-a609-332b139e7327 None None] MFA Rules not processed for user `771b1daa27f54605a1f8628050738e9e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:07.907743 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-a42f473c-5f78-4dcc-a609-332b139e7327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:07.951684 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fbb1f1-eb07-46fe-9fed-7309b06acdc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:07.952362 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fbb1f1-eb07-46fe-9fed-7309b06acdc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:07.952643 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fbb1f1-eb07-46fe-9fed-7309b06acdc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:07.952918 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fbb1f1-eb07-46fe-9fed-7309b06acdc1 admin admin] PATH_INFO: `/v3/projects/6a8e0f952ee540fdb963c8b5c891042b` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:07.954238 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fbb1f1-eb07-46fe-9fed-7309b06acdc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a8e0f952ee540fdb963c8b5c891042b)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:07.957116 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fbb1f1-eb07-46fe-9fed-7309b06acdc1 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:07.993663 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 945/1895] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:07 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:52:08.008457 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f473c-5f78-4dcc-a609-332b139e7327 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:08.010575 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-a42f473c-5f78-4dcc-a609-332b139e7327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:08.016783 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:08.020831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-e9fbb1f1-eb07-46fe-9fed-7309b06acdc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a8e0f952ee540fdb963c8b5c891042b', 'request_id': 'req-e9fbb1f1-eb07-46fe-9fed-7309b06acdc1'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:08.029034 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-e9fbb1f1-eb07-46fe-9fed-7309b06acdc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a8e0f952ee540fdb963c8b5c891042b', 'request_id': 'req-e9fbb1f1-eb07-46fe-9fed-7309b06acdc1'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:08.039102 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-e9fbb1f1-eb07-46fe-9fed-7309b06acdc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a8e0f952ee540fdb963c8b5c891042b', 'request_id': 'req-e9fbb1f1-eb07-46fe-9fed-7309b06acdc1'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:08.050626 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-e9fbb1f1-eb07-46fe-9fed-7309b06acdc1 admin admin] The token cache is being invalidate because project 6a8e0f952ee540fdb963c8b5c891042b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:52:08.051007 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-e9fbb1f1-eb07-46fe-9fed-7309b06acdc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:08.051758 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-e9fbb1f1-eb07-46fe-9fed-7309b06acdc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:08.054406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 951/1896] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:52:07 2022] DELETE /identity/v3/projects/6a8e0f952ee540fdb963c8b5c891042b => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:08.059421 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee3387-b508-4aab-8c39-e8030ee5417a None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:08.060903 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee3387-b508-4aab-8c39-e8030ee5417a None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:08.061179 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee3387-b508-4aab-8c39-e8030ee5417a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:08.083365 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-e0ee3387-b508-4aab-8c39-e8030ee5417a None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:08.086470 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-e0ee3387-b508-4aab-8c39-e8030ee5417a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:08.086960 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f473c-5f78-4dcc-a609-332b139e7327 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:08.165733 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e3439a-db30-4f46-8c40-586049281afb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:08.166371 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3439a-db30-4f46-8c40-586049281afb service cinder] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:08.166657 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3439a-db30-4f46-8c40-586049281afb service cinder] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:08.167073 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3439a-db30-4f46-8c40-586049281afb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:08.169486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-f1e3439a-db30-4f46-8c40-586049281afb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:08.173809 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 952/1897] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:08.182680 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c39fc9-77ec-43ed-b476-dada0ba3fe3b None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:08.183021 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c39fc9-77ec-43ed-b476-dada0ba3fe3b None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:08.183253 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c39fc9-77ec-43ed-b476-dada0ba3fe3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:08.208023 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-d5c39fc9-77ec-43ed-b476-dada0ba3fe3b None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:08.209779 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-d5c39fc9-77ec-43ed-b476-dada0ba3fe3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:08.219076 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 953/1898] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:52:08.223833 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c39fc9-77ec-43ed-b476-dada0ba3fe3b None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:08.228355 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:08.234462 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c39fc9-77ec-43ed-b476-dada0ba3fe3b None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:08.241839 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae683c-63ed-47c2-ab01-27ef2ae59d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:08.242448 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae683c-63ed-47c2-ab01-27ef2ae59d90 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:08.242815 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae683c-63ed-47c2-ab01-27ef2ae59d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:08.243009 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae683c-63ed-47c2-ab01-27ef2ae59d90 admin admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:08.243786 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ae683c-63ed-47c2-ab01-27ef2ae59d90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:52:08.244128 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ae683c-63ed-47c2-ab01-27ef2ae59d90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:08.245887 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ae683c-63ed-47c2-ab01-27ef2ae59d90 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:08.266896 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 954/1899] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:52:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:52:08.267084 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e3439a-db30-4f46-8c40-586049281afb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:08.267084 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e3439a-db30-4f46-8c40-586049281afb service cinder] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:08.267906 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae683c-63ed-47c2-ab01-27ef2ae59d90 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:08.280583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae683c-63ed-47c2-ab01-27ef2ae59d90 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:08.286610 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 946/1900] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:52:08 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:52:08.286713 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a224c2-066b-4271-a8f5-bf1d1d8627ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:08.286713 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a224c2-066b-4271-a8f5-bf1d1d8627ad admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:08.286713 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a224c2-066b-4271-a8f5-bf1d1d8627ad admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:08.286966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a224c2-066b-4271-a8f5-bf1d1d8627ad admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:08.287526 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a224c2-066b-4271-a8f5-bf1d1d8627ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:08.289516 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a224c2-066b-4271-a8f5-bf1d1d8627ad admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:08.327688 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 955/1901] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:52:08 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:52:08.331242 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e3439a-db30-4f46-8c40-586049281afb service cinder] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:08.347176 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e3439a-db30-4f46-8c40-586049281afb service cinder] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:08.465213 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e56829-2240-491b-aa9f-f34c92b92346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:08.466134 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e56829-2240-491b-aa9f-f34c92b92346 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:08.466519 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e56829-2240-491b-aa9f-f34c92b92346 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:08.466911 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e56829-2240-491b-aa9f-f34c92b92346 admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:08.467689 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e56829-2240-491b-aa9f-f34c92b92346 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:08.472961 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e56829-2240-491b-aa9f-f34c92b92346 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:08.507392 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 947/1902] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:52:08 2022] POST /identity/v3/users => generated 287 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:52:08.508667 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a224c2-066b-4271-a8f5-bf1d1d8627ad admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:08.522099 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a224c2-066b-4271-a8f5-bf1d1d8627ad admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:08.529000 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5c99e3-812f-4f6e-b532-8c283bda63ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:08.529994 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5c99e3-812f-4f6e-b532-8c283bda63ce admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:08.530715 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5c99e3-812f-4f6e-b532-8c283bda63ce admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:08.531184 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5c99e3-812f-4f6e-b532-8c283bda63ce admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:08.532267 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5c99e3-812f-4f6e-b532-8c283bda63ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:08.534079 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5c99e3-812f-4f6e-b532-8c283bda63ce admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:08.551993 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 956/1903] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:08 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:52:08.552477 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e56829-2240-491b-aa9f-f34c92b92346 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:08.565390 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e56829-2240-491b-aa9f-f34c92b92346 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:08.572105 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fbeeac-21a4-436d-bb57-a7af2e4b61e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:08.572951 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fbeeac-21a4-436d-bb57-a7af2e4b61e2 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:08.573062 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fbeeac-21a4-436d-bb57-a7af2e4b61e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:08.573285 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fbeeac-21a4-436d-bb57-a7af2e4b61e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:08.573995 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fbeeac-21a4-436d-bb57-a7af2e4b61e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:08.575623 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fbeeac-21a4-436d-bb57-a7af2e4b61e2 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:08.591253 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 948/1904] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 12:52:08.592656 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5c99e3-812f-4f6e-b532-8c283bda63ce admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:08.613637 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5c99e3-812f-4f6e-b532-8c283bda63ce admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:08.620589 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed055f-fbbe-4a80-853f-83ec22b21d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:08.621370 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed055f-fbbe-4a80-853f-83ec22b21d08 admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:08.621748 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed055f-fbbe-4a80-853f-83ec22b21d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:08.622119 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed055f-fbbe-4a80-853f-83ec22b21d08 admin admin] PATH_INFO: `/v3/projects/ff00ee711e2b42b9beaf866b10cc8f6f/users/c6f439a6212643248d050040d4eac29a/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:08.642429 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ed055f-fbbe-4a80-853f-83ec22b21d08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff00ee711e2b42b9beaf866b10cc8f6f, user_id=c6f439a6212643248d050040d4eac29a, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:08.645756 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ed055f-fbbe-4a80-853f-83ec22b21d08 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:08.668396 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 957/1905] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:52:08 2022] PUT /identity/v3/projects/ff00ee711e2b42b9beaf866b10cc8f6f/users/c6f439a6212643248d050040d4eac29a/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:08.675558 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c2519-99bb-4961-b913-1f08dfa3cb90 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:08.676442 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c2519-99bb-4961-b913-1f08dfa3cb90 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:08.676921 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c2519-99bb-4961-b913-1f08dfa3cb90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:08.729639 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-0d7c2519-99bb-4961-b913-1f08dfa3cb90 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:08.731571 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-0d7c2519-99bb-4961-b913-1f08dfa3cb90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:08.809748 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 949/1906] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:52:08.820112 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed055f-fbbe-4a80-853f-83ec22b21d08 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:08.822105 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-22ed055f-fbbe-4a80-853f-83ec22b21d08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:08.944644 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed055f-fbbe-4a80-853f-83ec22b21d08 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:08.950858 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54baee67-7b0a-4cbb-b355-c68745fece29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:08.950858 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54baee67-7b0a-4cbb-b355-c68745fece29 service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:08.950858 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54baee67-7b0a-4cbb-b355-c68745fece29 service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:08.951188 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54baee67-7b0a-4cbb-b355-c68745fece29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:08.962863 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54baee67-7b0a-4cbb-b355-c68745fece29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:08.965094 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54baee67-7b0a-4cbb-b355-c68745fece29 service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:08.972134 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:08.981188 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 958/1907] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:52:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:52:09.115300 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7c2519-99bb-4961-b913-1f08dfa3cb90 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:09.117028 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-0d7c2519-99bb-4961-b913-1f08dfa3cb90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:09.139615 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7c2519-99bb-4961-b913-1f08dfa3cb90 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:09.144601 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99c91b-8242-474f-8b12-41fa2a791db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:09.145242 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99c91b-8242-474f-8b12-41fa2a791db9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:09.145706 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99c91b-8242-474f-8b12-41fa2a791db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:09.145820 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99c91b-8242-474f-8b12-41fa2a791db9 admin admin] PATH_INFO: `/v3/users/fbffc2026f394be2a1b9c54ce9b96f79` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:09.147472 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c99c91b-8242-474f-8b12-41fa2a791db9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbffc2026f394be2a1b9c54ce9b96f79)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:09.149591 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c99c91b-8242-474f-8b12-41fa2a791db9 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:09.209913 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-8c99c91b-8242-474f-8b12-41fa2a791db9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbffc2026f394be2a1b9c54ce9b96f79'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:09.210584 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-8c99c91b-8242-474f-8b12-41fa2a791db9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbffc2026f394be2a1b9c54ce9b96f79'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:09.211588 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-8c99c91b-8242-474f-8b12-41fa2a791db9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbffc2026f394be2a1b9c54ce9b96f79'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:09.219751 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:09.234428 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-8c99c91b-8242-474f-8b12-41fa2a791db9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbffc2026f394be2a1b9c54ce9b96f79'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:09.246151 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-8c99c91b-8242-474f-8b12-41fa2a791db9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbffc2026f394be2a1b9c54ce9b96f79'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:09.263021 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 950/1908] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:52:09 2022] DELETE /identity/v3/users/fbffc2026f394be2a1b9c54ce9b96f79 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:09.267521 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452401f-03d4-489b-85fa-a45604a0f5f3 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:09.272287 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452401f-03d4-489b-85fa-a45604a0f5f3 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:09.272571 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452401f-03d4-489b-85fa-a45604a0f5f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:09.297177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-8452401f-03d4-489b-85fa-a45604a0f5f3 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:09.298435 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-8452401f-03d4-489b-85fa-a45604a0f5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:09.376153 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 959/1909] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:52:09.386475 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99c91b-8242-474f-8b12-41fa2a791db9 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:09.388119 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-8c99c91b-8242-474f-8b12-41fa2a791db9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:09.483202 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99c91b-8242-474f-8b12-41fa2a791db9 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:09.486994 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb3aee4-28ab-44c5-87bc-44939cb728a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:09.487612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb3aee4-28ab-44c5-87bc-44939cb728a2 service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:09.487960 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb3aee4-28ab-44c5-87bc-44939cb728a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:09.488213 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb3aee4-28ab-44c5-87bc-44939cb728a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:09.500639 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb3aee4-28ab-44c5-87bc-44939cb728a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:09.502397 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb3aee4-28ab-44c5-87bc-44939cb728a2 service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:09.514404 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 951/1910] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:52:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:52:09.766251 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109acde6-1e48-4008-acbd-1ded2b3212c4 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:09.766770 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109acde6-1e48-4008-acbd-1ded2b3212c4 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:09.766886 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109acde6-1e48-4008-acbd-1ded2b3212c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:09.814356 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-109acde6-1e48-4008-acbd-1ded2b3212c4 None None] MFA Rules not processed for user `c6f439a6212643248d050040d4eac29a`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:09.815973 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-109acde6-1e48-4008-acbd-1ded2b3212c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:09.915060 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 960/1911] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:09.920217 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb3aee4-28ab-44c5-87bc-44939cb728a2 service neutron] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:09.922417 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-1eb3aee4-28ab-44c5-87bc-44939cb728a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:09.939465 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109acde6-1e48-4008-acbd-1ded2b3212c4 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:09.939465 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-109acde6-1e48-4008-acbd-1ded2b3212c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:09.941611 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb3aee4-28ab-44c5-87bc-44939cb728a2 service neutron] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:09.954607 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7e247f-030e-4b27-bc9f-6fb60b4ccee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:09.955337 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e247f-030e-4b27-bc9f-6fb60b4ccee4 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:09.955880 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e247f-030e-4b27-bc9f-6fb60b4ccee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:09.956229 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e247f-030e-4b27-bc9f-6fb60b4ccee4 admin admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:09.957012 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7e247f-030e-4b27-bc9f-6fb60b4ccee4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:09.959772 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7e247f-030e-4b27-bc9f-6fb60b4ccee4 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:09.960302 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109acde6-1e48-4008-acbd-1ded2b3212c4 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:09.968122 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7627a2b6-619f-435c-a7d7-e5e3da701fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:09.968804 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627a2b6-619f-435c-a7d7-e5e3da701fdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:09.969076 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627a2b6-619f-435c-a7d7-e5e3da701fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:09.969351 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627a2b6-619f-435c-a7d7-e5e3da701fdb admin admin] PATH_INFO: `/v3/projects/c82f5e83a6804126bcef77e595ed076f` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:09.971200 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7627a2b6-619f-435c-a7d7-e5e3da701fdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c82f5e83a6804126bcef77e595ed076f)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:09.975611 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7627a2b6-619f-435c-a7d7-e5e3da701fdb admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:09.995571 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 952/1912] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:52:09 2022] POST /identity/v3/projects => generated 369 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:52:09.999994 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7e247f-030e-4b27-bc9f-6fb60b4ccee4 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:10.039215 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-7627a2b6-619f-435c-a7d7-e5e3da701fdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c82f5e83a6804126bcef77e595ed076f', 'request_id': 'req-7627a2b6-619f-435c-a7d7-e5e3da701fdb'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:10.045526 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-7627a2b6-619f-435c-a7d7-e5e3da701fdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c82f5e83a6804126bcef77e595ed076f', 'request_id': 'req-7627a2b6-619f-435c-a7d7-e5e3da701fdb'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:10.053698 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-7627a2b6-619f-435c-a7d7-e5e3da701fdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c82f5e83a6804126bcef77e595ed076f', 'request_id': 'req-7627a2b6-619f-435c-a7d7-e5e3da701fdb'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:10.064855 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-7627a2b6-619f-435c-a7d7-e5e3da701fdb admin admin] The token cache is being invalidate because project c82f5e83a6804126bcef77e595ed076f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:52:10.065149 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-7627a2b6-619f-435c-a7d7-e5e3da701fdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:10.065804 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-7627a2b6-619f-435c-a7d7-e5e3da701fdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:10.067361 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 961/1913] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:52:09 2022] DELETE /identity/v3/projects/c82f5e83a6804126bcef77e595ed076f => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:52:10.071763 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7627a2b6-619f-435c-a7d7-e5e3da701fdb admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:10.072648 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7e247f-030e-4b27-bc9f-6fb60b4ccee4 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:10.073674 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-7627a2b6-619f-435c-a7d7-e5e3da701fdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:10.087870 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:10.135179 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:10.156050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c4751-d92d-499a-92ff-5e7bd9001080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:10.157543 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c4751-d92d-499a-92ff-5e7bd9001080 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:10.164316 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c4751-d92d-499a-92ff-5e7bd9001080 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:10.165431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c4751-d92d-499a-92ff-5e7bd9001080 admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:10.174479 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9c4751-d92d-499a-92ff-5e7bd9001080 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:10.177408 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9c4751-d92d-499a-92ff-5e7bd9001080 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:10.200373 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7627a2b6-619f-435c-a7d7-e5e3da701fdb admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:10.205342 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57359ef2-7cca-497d-b2ce-e0355bbf8aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:10.206978 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57359ef2-7cca-497d-b2ce-e0355bbf8aa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:10.206978 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57359ef2-7cca-497d-b2ce-e0355bbf8aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:10.206978 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57359ef2-7cca-497d-b2ce-e0355bbf8aa5 admin admin] PATH_INFO: `/v3/users/771b1daa27f54605a1f8628050738e9e` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:10.207260 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 953/1914] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:52:09 2022] POST /identity/v3/users => generated 291 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:52:10.208276 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57359ef2-7cca-497d-b2ce-e0355bbf8aa5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=771b1daa27f54605a1f8628050738e9e)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:10.211048 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57359ef2-7cca-497d-b2ce-e0355bbf8aa5 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:10.211438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c4751-d92d-499a-92ff-5e7bd9001080 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:10.213588 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-bf9c4751-d92d-499a-92ff-5e7bd9001080 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:10.227439 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c4751-d92d-499a-92ff-5e7bd9001080 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:10.232994 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3623fd94-d6bd-4c4e-9fa9-ad0911fe7a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:10.233624 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623fd94-d6bd-4c4e-9fa9-ad0911fe7a20 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:10.233907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623fd94-d6bd-4c4e-9fa9-ad0911fe7a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:10.234184 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623fd94-d6bd-4c4e-9fa9-ad0911fe7a20 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:10.234900 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3623fd94-d6bd-4c4e-9fa9-ad0911fe7a20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:10.236667 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3623fd94-d6bd-4c4e-9fa9-ad0911fe7a20 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:10.250028 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 954/1915] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 12:52:10.254295 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3623fd94-d6bd-4c4e-9fa9-ad0911fe7a20 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:10.267455 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3623fd94-d6bd-4c4e-9fa9-ad0911fe7a20 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:10.270560 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-57359ef2-7cca-497d-b2ce-e0355bbf8aa5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '771b1daa27f54605a1f8628050738e9e'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:10.274074 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a82d18-8d36-434a-b30c-3e17118b1fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:10.275081 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a82d18-8d36-434a-b30c-3e17118b1fdb admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:10.275433 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a82d18-8d36-434a-b30c-3e17118b1fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:10.275947 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a82d18-8d36-434a-b30c-3e17118b1fdb admin admin] PATH_INFO: `/v3/projects/761b59a0b7184c2c814c83c78986ec0d/users/59deeab927c44656bd4ff7adaacf762a/roles/0e78444e43c744668178971af4087ca0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:10.281664 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-57359ef2-7cca-497d-b2ce-e0355bbf8aa5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '771b1daa27f54605a1f8628050738e9e'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:10.282363 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-57359ef2-7cca-497d-b2ce-e0355bbf8aa5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '771b1daa27f54605a1f8628050738e9e'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:10.292464 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-57359ef2-7cca-497d-b2ce-e0355bbf8aa5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '771b1daa27f54605a1f8628050738e9e'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:10.295018 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a82d18-8d36-434a-b30c-3e17118b1fdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=761b59a0b7184c2c814c83c78986ec0d, user_id=59deeab927c44656bd4ff7adaacf762a, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:10.298304 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a82d18-8d36-434a-b30c-3e17118b1fdb admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:10.312060 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-57359ef2-7cca-497d-b2ce-e0355bbf8aa5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '771b1daa27f54605a1f8628050738e9e'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:10.319716 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 955/1916] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:52:10 2022] PUT /identity/v3/projects/761b59a0b7184c2c814c83c78986ec0d/users/59deeab927c44656bd4ff7adaacf762a/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:10.320100 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 962/1917] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:52:10 2022] DELETE /identity/v3/users/771b1daa27f54605a1f8628050738e9e => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:52:10.331922 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a82d18-8d36-434a-b30c-3e17118b1fdb admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:10.336398 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-d5a82d18-8d36-434a-b30c-3e17118b1fdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:10.433683 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a82d18-8d36-434a-b30c-3e17118b1fdb admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:10.628095 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc304d-01ce-4c03-9b3f-a82b5a69201a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:10.628821 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc304d-01ce-4c03-9b3f-a82b5a69201a admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:10.629110 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc304d-01ce-4c03-9b3f-a82b5a69201a admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:10.630111 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc304d-01ce-4c03-9b3f-a82b5a69201a admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:10.630977 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfc304d-01ce-4c03-9b3f-a82b5a69201a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:10.634842 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfc304d-01ce-4c03-9b3f-a82b5a69201a admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:10.679552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57359ef2-7cca-497d-b2ce-e0355bbf8aa5 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:10.682623 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 956/1918] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:10 2022] GET /identity/v3/roles => generated 1420 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 12:52:10.704614 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57359ef2-7cca-497d-b2ce-e0355bbf8aa5 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:10.720568 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f4334d-6726-4d79-9559-5de429590e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:10.721368 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f4334d-6726-4d79-9559-5de429590e81 admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:10.721673 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f4334d-6726-4d79-9559-5de429590e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:10.721912 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f4334d-6726-4d79-9559-5de429590e81 admin admin] PATH_INFO: `/v3/projects/761b59a0b7184c2c814c83c78986ec0d/users/59deeab927c44656bd4ff7adaacf762a/roles/0e78444e43c744668178971af4087ca0` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:10.728311 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f4334d-6726-4d79-9559-5de429590e81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=761b59a0b7184c2c814c83c78986ec0d, user_id=59deeab927c44656bd4ff7adaacf762a, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:10.733488 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f4334d-6726-4d79-9559-5de429590e81 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:10.764076 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 963/1919] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:52:10 2022] PUT /identity/v3/projects/761b59a0b7184c2c814c83c78986ec0d/users/59deeab927c44656bd4ff7adaacf762a/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:11.069797 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc304d-01ce-4c03-9b3f-a82b5a69201a admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:11.072680 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-8bfc304d-01ce-4c03-9b3f-a82b5a69201a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:11.125104 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc304d-01ce-4c03-9b3f-a82b5a69201a admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:11.220682 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:11.247061 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b444e2c-1eb9-4d13-a2e8-50f1e26f20fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:11.248516 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b444e2c-1eb9-4d13-a2e8-50f1e26f20fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:11.248905 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b444e2c-1eb9-4d13-a2e8-50f1e26f20fc admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:11.249275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b444e2c-1eb9-4d13-a2e8-50f1e26f20fc admin admin] PATH_INFO: `/v3/projects/0593ebe7f9104536b9723b63df5f08ac` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:11.251315 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b444e2c-1eb9-4d13-a2e8-50f1e26f20fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=0593ebe7f9104536b9723b63df5f08ac)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:11.253661 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b444e2c-1eb9-4d13-a2e8-50f1e26f20fc admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:11.336853 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-3b444e2c-1eb9-4d13-a2e8-50f1e26f20fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0593ebe7f9104536b9723b63df5f08ac', 'request_id': 'req-3b444e2c-1eb9-4d13-a2e8-50f1e26f20fc'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:11.351452 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-3b444e2c-1eb9-4d13-a2e8-50f1e26f20fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0593ebe7f9104536b9723b63df5f08ac', 'request_id': 'req-3b444e2c-1eb9-4d13-a2e8-50f1e26f20fc'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:11.371041 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-3b444e2c-1eb9-4d13-a2e8-50f1e26f20fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0593ebe7f9104536b9723b63df5f08ac', 'request_id': 'req-3b444e2c-1eb9-4d13-a2e8-50f1e26f20fc'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:11.392997 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-3b444e2c-1eb9-4d13-a2e8-50f1e26f20fc admin admin] The token cache is being invalidate because project 0593ebe7f9104536b9723b63df5f08ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:52:11.393295 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-3b444e2c-1eb9-4d13-a2e8-50f1e26f20fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:11.394095 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-3b444e2c-1eb9-4d13-a2e8-50f1e26f20fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:11.397364 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 957/1920] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:52:11 2022] DELETE /identity/v3/projects/0593ebe7f9104536b9723b63df5f08ac => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:11.402788 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae868b-2083-4c8d-bebf-034dbac1b740 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:11.403296 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae868b-2083-4c8d-bebf-034dbac1b740 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:11.403390 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae868b-2083-4c8d-bebf-034dbac1b740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:11.433559 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-d4ae868b-2083-4c8d-bebf-034dbac1b740 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:11.434798 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-d4ae868b-2083-4c8d-bebf-034dbac1b740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:11.524640 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 964/1921] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:11.534194 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c9b6d-0f1b-4767-ad2c-7dbef5f074ef None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:11.534583 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c9b6d-0f1b-4767-ad2c-7dbef5f074ef None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:11.534831 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c9b6d-0f1b-4767-ad2c-7dbef5f074ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:11.562166 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-c88c9b6d-0f1b-4767-ad2c-7dbef5f074ef None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:11.563702 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-c88c9b6d-0f1b-4767-ad2c-7dbef5f074ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:11.593280 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 958/1922] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:11.597341 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ae868b-2083-4c8d-bebf-034dbac1b740 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:11.618108 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ae868b-2083-4c8d-bebf-034dbac1b740 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:11.624983 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339caf54-46f3-4e1f-8daa-9cc721f0048b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:11.625686 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339caf54-46f3-4e1f-8daa-9cc721f0048b admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:11.626052 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339caf54-46f3-4e1f-8daa-9cc721f0048b admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:11.626374 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339caf54-46f3-4e1f-8daa-9cc721f0048b admin admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:11.627192 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339caf54-46f3-4e1f-8daa-9cc721f0048b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:52:11.627528 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339caf54-46f3-4e1f-8daa-9cc721f0048b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:11.629637 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339caf54-46f3-4e1f-8daa-9cc721f0048b admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:11.648136 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 965/1923] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:52:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:52:11.650144 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c9b6d-0f1b-4767-ad2c-7dbef5f074ef None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:11.659883 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c9b6d-0f1b-4767-ad2c-7dbef5f074ef None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:11.666242 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cd60ac-01e4-449c-8970-236ae3ab4baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:11.666929 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd60ac-01e4-449c-8970-236ae3ab4baa admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:11.667823 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd60ac-01e4-449c-8970-236ae3ab4baa admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:11.667823 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd60ac-01e4-449c-8970-236ae3ab4baa admin admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:11.668240 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cd60ac-01e4-449c-8970-236ae3ab4baa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:11.670279 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cd60ac-01e4-449c-8970-236ae3ab4baa admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:11.712493 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 959/1924] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:52:11 2022] POST /identity/v3/projects => generated 385 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:52:11.714314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339caf54-46f3-4e1f-8daa-9cc721f0048b admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:11.728364 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339caf54-46f3-4e1f-8daa-9cc721f0048b admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:11.732746 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:11.817837 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d34313-7d9a-4771-91e6-3f879d845321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:11.819487 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d34313-7d9a-4771-91e6-3f879d845321 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:11.819757 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d34313-7d9a-4771-91e6-3f879d845321 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:11.820073 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d34313-7d9a-4771-91e6-3f879d845321 admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:11.821680 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d34313-7d9a-4771-91e6-3f879d845321 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:11.824420 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d34313-7d9a-4771-91e6-3f879d845321 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:11.849359 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 966/1925] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:52:11 2022] POST /identity/v3/users => generated 293 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:52:11.854488 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cd60ac-01e4-449c-8970-236ae3ab4baa admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:11.866644 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cd60ac-01e4-449c-8970-236ae3ab4baa admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:11.871760 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef454846-1028-4fd9-8e2d-b82334f0623e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:11.873085 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef454846-1028-4fd9-8e2d-b82334f0623e admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:11.873759 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef454846-1028-4fd9-8e2d-b82334f0623e admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:11.874041 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef454846-1028-4fd9-8e2d-b82334f0623e admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:11.874747 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef454846-1028-4fd9-8e2d-b82334f0623e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:11.877303 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef454846-1028-4fd9-8e2d-b82334f0623e admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:11.889187 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 960/1926] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:52:11.893026 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d34313-7d9a-4771-91e6-3f879d845321 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:11.907913 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d34313-7d9a-4771-91e6-3f879d845321 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:11.913946 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab2c9c6-0b72-4c7c-81e0-7f88c1fbef39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:11.914807 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2c9c6-0b72-4c7c-81e0-7f88c1fbef39 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:11.915087 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2c9c6-0b72-4c7c-81e0-7f88c1fbef39 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:11.915354 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2c9c6-0b72-4c7c-81e0-7f88c1fbef39 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:11.916098 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab2c9c6-0b72-4c7c-81e0-7f88c1fbef39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:11.917710 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab2c9c6-0b72-4c7c-81e0-7f88c1fbef39 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:11.929754 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 967/1927] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 12:52:11.933811 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef454846-1028-4fd9-8e2d-b82334f0623e admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:11.947911 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef454846-1028-4fd9-8e2d-b82334f0623e admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:11.954130 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2851098c-1fa4-4e6d-9777-133417d7a4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:11.956210 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2851098c-1fa4-4e6d-9777-133417d7a4c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:11.956210 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2851098c-1fa4-4e6d-9777-133417d7a4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:11.956416 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2851098c-1fa4-4e6d-9777-133417d7a4c4 admin admin] PATH_INFO: `/v3/projects/172b4d6df75c43e79da9682076f27c45/users/b5be28194acc47ed9dd469b22f7067a4/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:11.979221 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2851098c-1fa4-4e6d-9777-133417d7a4c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=172b4d6df75c43e79da9682076f27c45, user_id=b5be28194acc47ed9dd469b22f7067a4, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:11.982004 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2851098c-1fa4-4e6d-9777-133417d7a4c4 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:11.999881 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 961/1928] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:52:11 2022] PUT /identity/v3/projects/172b4d6df75c43e79da9682076f27c45/users/b5be28194acc47ed9dd469b22f7067a4/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:12.014031 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c70cf-6ddc-47ea-b2de-923c31c9ddcd None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:12.016315 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c70cf-6ddc-47ea-b2de-923c31c9ddcd None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:12.017619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c70cf-6ddc-47ea-b2de-923c31c9ddcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:12.065277 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-547c70cf-6ddc-47ea-b2de-923c31c9ddcd None None] MFA Rules not processed for user `b5be28194acc47ed9dd469b22f7067a4`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:12.066729 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-547c70cf-6ddc-47ea-b2de-923c31c9ddcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:12.066846 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16f595-d06c-42a8-8f9e-e9edce888f34 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:12.067049 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16f595-d06c-42a8-8f9e-e9edce888f34 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:12.067331 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16f595-d06c-42a8-8f9e-e9edce888f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:12.161720 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 968/1929] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:12.176673 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c70cf-6ddc-47ea-b2de-923c31c9ddcd None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:12.185531 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-547c70cf-6ddc-47ea-b2de-923c31c9ddcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:12.208124 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-2d16f595-d06c-42a8-8f9e-e9edce888f34 None None] MFA Rules not processed for user `59deeab927c44656bd4ff7adaacf762a`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:12.210166 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-2d16f595-d06c-42a8-8f9e-e9edce888f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:12.273848 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c70cf-6ddc-47ea-b2de-923c31c9ddcd None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:12.278799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978a3b6b-a43b-4746-ae12-d294a6accabd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:12.279732 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a3b6b-a43b-4746-ae12-d294a6accabd service cinder] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:12.280024 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a3b6b-a43b-4746-ae12-d294a6accabd service cinder] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:12.280308 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a3b6b-a43b-4746-ae12-d294a6accabd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:12.291259 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978a3b6b-a43b-4746-ae12-d294a6accabd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:12.292822 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978a3b6b-a43b-4746-ae12-d294a6accabd service cinder] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:12.305945 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 969/1930] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:52:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 12:52:12.337792 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978a3b6b-a43b-4746-ae12-d294a6accabd service cinder] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:12.398418 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:12.415913 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 962/1931] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:12 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:12.429739 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d16f595-d06c-42a8-8f9e-e9edce888f34 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:12.431477 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-2d16f595-d06c-42a8-8f9e-e9edce888f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:12.455299 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d16f595-d06c-42a8-8f9e-e9edce888f34 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:12.462204 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7e9164-c02d-4cee-a988-12c03574293f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:12.463034 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e9164-c02d-4cee-a988-12c03574293f service cinder] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:12.463456 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e9164-c02d-4cee-a988-12c03574293f service cinder] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:12.463915 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e9164-c02d-4cee-a988-12c03574293f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:12.465889 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-5b7e9164-c02d-4cee-a988-12c03574293f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:12.473615 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:12.519076 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978a3b6b-a43b-4746-ae12-d294a6accabd service cinder] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:12.529734 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb33ea-86f3-4f04-b85d-f1e412c13dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:12.531744 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb33ea-86f3-4f04-b85d-f1e412c13dbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:12.532459 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb33ea-86f3-4f04-b85d-f1e412c13dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:12.532842 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb33ea-86f3-4f04-b85d-f1e412c13dbe admin admin] PATH_INFO: `/v3/users/b5be28194acc47ed9dd469b22f7067a4` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:12.534626 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fb33ea-86f3-4f04-b85d-f1e412c13dbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5be28194acc47ed9dd469b22f7067a4)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:12.539820 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fb33ea-86f3-4f04-b85d-f1e412c13dbe admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:12.618246 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7e9164-c02d-4cee-a988-12c03574293f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:12.620533 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7e9164-c02d-4cee-a988-12c03574293f service cinder] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:12.626991 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-81fb33ea-86f3-4f04-b85d-f1e412c13dbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5be28194acc47ed9dd469b22f7067a4'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:12.634566 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 963/1932] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:52:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:52:12.637821 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-81fb33ea-86f3-4f04-b85d-f1e412c13dbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5be28194acc47ed9dd469b22f7067a4'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:12.638627 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-81fb33ea-86f3-4f04-b85d-f1e412c13dbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5be28194acc47ed9dd469b22f7067a4'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:12.650583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-81fb33ea-86f3-4f04-b85d-f1e412c13dbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5be28194acc47ed9dd469b22f7067a4'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:12.674129 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-81fb33ea-86f3-4f04-b85d-f1e412c13dbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5be28194acc47ed9dd469b22f7067a4'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:12.685857 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 970/1933] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:52:12 2022] DELETE /identity/v3/users/b5be28194acc47ed9dd469b22f7067a4 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:52:12.695806 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d31d92-8aed-41bc-bc4f-17969bfd18e6 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:12.696254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d31d92-8aed-41bc-bc4f-17969bfd18e6 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:12.696534 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d31d92-8aed-41bc-bc4f-17969bfd18e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:12.737476 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-28d31d92-8aed-41bc-bc4f-17969bfd18e6 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:12.738906 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-28d31d92-8aed-41bc-bc4f-17969bfd18e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:12.920850 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 964/1934] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:12.929228 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb33ea-86f3-4f04-b85d-f1e412c13dbe admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:12.931242 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-81fb33ea-86f3-4f04-b85d-f1e412c13dbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:13.059846 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb33ea-86f3-4f04-b85d-f1e412c13dbe admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:13.064294 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6e4e5-af21-44b5-a4e5-332e914f1d4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:13.065083 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6e4e5-af21-44b5-a4e5-332e914f1d4c service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:13.065397 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6e4e5-af21-44b5-a4e5-332e914f1d4c service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:13.065738 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6e4e5-af21-44b5-a4e5-332e914f1d4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:13.077046 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a6e4e5-af21-44b5-a4e5-332e914f1d4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:13.078679 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a6e4e5-af21-44b5-a4e5-332e914f1d4c service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:13.091507 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 971/1935] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:52:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:52:13.552120 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d31d92-8aed-41bc-bc4f-17969bfd18e6 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:13.555361 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-28d31d92-8aed-41bc-bc4f-17969bfd18e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:13.572130 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d31d92-8aed-41bc-bc4f-17969bfd18e6 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:13.578293 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a269eb7-1e55-4807-8edb-6ca2d0d2a90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:13.579176 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a269eb7-1e55-4807-8edb-6ca2d0d2a90a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:13.579542 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a269eb7-1e55-4807-8edb-6ca2d0d2a90a admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:13.586158 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a269eb7-1e55-4807-8edb-6ca2d0d2a90a admin admin] PATH_INFO: `/v3/projects/172b4d6df75c43e79da9682076f27c45` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:13.603034 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a269eb7-1e55-4807-8edb-6ca2d0d2a90a admin admin] RBAC: Authorizing `identity:delete_project(project_id=172b4d6df75c43e79da9682076f27c45)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:13.607847 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a269eb7-1e55-4807-8edb-6ca2d0d2a90a admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:13.671438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-8a269eb7-1e55-4807-8edb-6ca2d0d2a90a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '172b4d6df75c43e79da9682076f27c45', 'request_id': 'req-8a269eb7-1e55-4807-8edb-6ca2d0d2a90a'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:13.682105 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-8a269eb7-1e55-4807-8edb-6ca2d0d2a90a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '172b4d6df75c43e79da9682076f27c45', 'request_id': 'req-8a269eb7-1e55-4807-8edb-6ca2d0d2a90a'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:13.704203 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-8a269eb7-1e55-4807-8edb-6ca2d0d2a90a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '172b4d6df75c43e79da9682076f27c45', 'request_id': 'req-8a269eb7-1e55-4807-8edb-6ca2d0d2a90a'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:13.725442 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-8a269eb7-1e55-4807-8edb-6ca2d0d2a90a admin admin] The token cache is being invalidate because project 172b4d6df75c43e79da9682076f27c45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:52:13.725735 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-8a269eb7-1e55-4807-8edb-6ca2d0d2a90a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:13.726432 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-8a269eb7-1e55-4807-8edb-6ca2d0d2a90a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:13.729312 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 965/1936] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:52:13 2022] DELETE /identity/v3/projects/172b4d6df75c43e79da9682076f27c45 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:13.734702 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b31f5-c9c9-4788-9b6f-24288b686c1b None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:13.743002 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b31f5-c9c9-4788-9b6f-24288b686c1b None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:13.743588 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b31f5-c9c9-4788-9b6f-24288b686c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:13.782063 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-817b31f5-c9c9-4788-9b6f-24288b686c1b None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:13.783499 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-817b31f5-c9c9-4788-9b6f-24288b686c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:13.822449 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:13.849924 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 972/1937] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:13.857720 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0809690-bb16-4fe4-a6c1-472d2d4e74b1 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:13.858011 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0809690-bb16-4fe4-a6c1-472d2d4e74b1 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:13.858318 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0809690-bb16-4fe4-a6c1-472d2d4e74b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:13.885666 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-b0809690-bb16-4fe4-a6c1-472d2d4e74b1 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:13.886884 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-b0809690-bb16-4fe4-a6c1-472d2d4e74b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:13.899670 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 966/1938] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:13.900344 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b31f5-c9c9-4788-9b6f-24288b686c1b None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:13.901818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-817b31f5-c9c9-4788-9b6f-24288b686c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:13.904738 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0809690-bb16-4fe4-a6c1-472d2d4e74b1 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:13.915478 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0809690-bb16-4fe4-a6c1-472d2d4e74b1 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:13.915901 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b31f5-c9c9-4788-9b6f-24288b686c1b None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:13.920574 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6085a8e3-f0c4-47df-91a5-09448d4e66f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:13.921325 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a51c69-1924-421f-965c-1f5694612999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:13.921449 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085a8e3-f0c4-47df-91a5-09448d4e66f4 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:13.921696 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085a8e3-f0c4-47df-91a5-09448d4e66f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:13.922008 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a51c69-1924-421f-965c-1f5694612999 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:13.922315 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a51c69-1924-421f-965c-1f5694612999 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:13.922523 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085a8e3-f0c4-47df-91a5-09448d4e66f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:13.922641 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a51c69-1924-421f-965c-1f5694612999 admin admin] PATH_INFO: `/v3/users/67de9c16b7a84740bbbfc63eab25c6e1` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:13.923468 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6085a8e3-f0c4-47df-91a5-09448d4e66f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:52:13.923797 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6085a8e3-f0c4-47df-91a5-09448d4e66f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:13.924539 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a51c69-1924-421f-965c-1f5694612999 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67de9c16b7a84740bbbfc63eab25c6e1)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:13.925552 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6085a8e3-f0c4-47df-91a5-09448d4e66f4 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:13.926568 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a51c69-1924-421f-965c-1f5694612999 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:13.939876 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 967/1939] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:52:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:52:13.944378 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6085a8e3-f0c4-47df-91a5-09448d4e66f4 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:13.956189 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6085a8e3-f0c4-47df-91a5-09448d4e66f4 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:13.961726 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b90fd55-93a0-4b45-8cea-e8079fe969a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:13.962356 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90fd55-93a0-4b45-8cea-e8079fe969a1 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:13.962916 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90fd55-93a0-4b45-8cea-e8079fe969a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:13.963354 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90fd55-93a0-4b45-8cea-e8079fe969a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:13.964213 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b90fd55-93a0-4b45-8cea-e8079fe969a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:13.966338 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b90fd55-93a0-4b45-8cea-e8079fe969a1 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:13.973652 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:13.988249 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-f3a51c69-1924-421f-965c-1f5694612999 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67de9c16b7a84740bbbfc63eab25c6e1'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:13.999682 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-f3a51c69-1924-421f-965c-1f5694612999 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67de9c16b7a84740bbbfc63eab25c6e1'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:14.000466 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-f3a51c69-1924-421f-965c-1f5694612999 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67de9c16b7a84740bbbfc63eab25c6e1'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:14.008843 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 968/1940] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:52:13 2022] POST /identity/v3/projects => generated 365 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:52:14.010091 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-f3a51c69-1924-421f-965c-1f5694612999 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67de9c16b7a84740bbbfc63eab25c6e1'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:14.013604 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b90fd55-93a0-4b45-8cea-e8079fe969a1 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:14.025591 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-f3a51c69-1924-421f-965c-1f5694612999 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67de9c16b7a84740bbbfc63eab25c6e1'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:14.032115 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 973/1941] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:52:13 2022] DELETE /identity/v3/users/67de9c16b7a84740bbbfc63eab25c6e1 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:14.036533 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d82ca0-004e-407f-9c92-a4f1a30ae922 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:14.036819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d82ca0-004e-407f-9c92-a4f1a30ae922 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:14.038010 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d82ca0-004e-407f-9c92-a4f1a30ae922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:14.058422 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-39d82ca0-004e-407f-9c92-a4f1a30ae922 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:14.059922 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-39d82ca0-004e-407f-9c92-a4f1a30ae922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:14.094980 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b90fd55-93a0-4b45-8cea-e8079fe969a1 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:14.132427 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 974/1942] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:52:14.142269 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d82ca0-004e-407f-9c92-a4f1a30ae922 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:14.143843 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-39d82ca0-004e-407f-9c92-a4f1a30ae922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:14.177031 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05d9d47-61c5-446d-9bd4-3c5b6d7b17f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:14.178105 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d9d47-61c5-446d-9bd4-3c5b6d7b17f0 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:14.178390 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d9d47-61c5-446d-9bd4-3c5b6d7b17f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:14.178666 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d9d47-61c5-446d-9bd4-3c5b6d7b17f0 admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:14.179387 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05d9d47-61c5-446d-9bd4-3c5b6d7b17f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:14.181618 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05d9d47-61c5-446d-9bd4-3c5b6d7b17f0 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:14.206483 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 969/1943] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:52:14 2022] POST /identity/v3/users => generated 283 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:52:14.211163 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05d9d47-61c5-446d-9bd4-3c5b6d7b17f0 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:14.213492 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-a05d9d47-61c5-446d-9bd4-3c5b6d7b17f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:14.215930 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d82ca0-004e-407f-9c92-a4f1a30ae922 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:14.221676 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2afe64-0766-4630-b4d0-9c9f25f03e39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:14.222589 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2afe64-0766-4630-b4d0-9c9f25f03e39 service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:14.223024 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2afe64-0766-4630-b4d0-9c9f25f03e39 service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:14.223590 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2afe64-0766-4630-b4d0-9c9f25f03e39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:14.227607 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05d9d47-61c5-446d-9bd4-3c5b6d7b17f0 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:14.232938 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:14.233356 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a3917-2515-481c-8825-410a2d4e097a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:14.233981 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a3917-2515-481c-8825-410a2d4e097a admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:14.234807 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a3917-2515-481c-8825-410a2d4e097a admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:14.234807 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a3917-2515-481c-8825-410a2d4e097a admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:14.235238 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4a3917-2515-481c-8825-410a2d4e097a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:14.236682 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2afe64-0766-4630-b4d0-9c9f25f03e39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:14.239031 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2afe64-0766-4630-b4d0-9c9f25f03e39 service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:14.241464 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4a3917-2515-481c-8825-410a2d4e097a admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:14.255578 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 975/1944] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:52:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 12:52:14.257661 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 970/1945] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 12:52:14.261969 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2afe64-0766-4630-b4d0-9c9f25f03e39 service neutron] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:14.272860 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2afe64-0766-4630-b4d0-9c9f25f03e39 service neutron] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:14.278241 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498cbe2c-a83a-46be-b08a-0b27dd4d6255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:14.278854 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498cbe2c-a83a-46be-b08a-0b27dd4d6255 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:14.279137 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498cbe2c-a83a-46be-b08a-0b27dd4d6255 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:14.279411 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498cbe2c-a83a-46be-b08a-0b27dd4d6255 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:14.280256 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498cbe2c-a83a-46be-b08a-0b27dd4d6255 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:14.281842 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498cbe2c-a83a-46be-b08a-0b27dd4d6255 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:14.293359 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 976/1946] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 12:52:14.297854 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a3917-2515-481c-8825-410a2d4e097a admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:14.310817 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a3917-2515-481c-8825-410a2d4e097a admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:14.316964 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2b27ad-a9b9-47f2-bbb3-c98e9c6a3363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:14.317731 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b27ad-a9b9-47f2-bbb3-c98e9c6a3363 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:14.318011 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b27ad-a9b9-47f2-bbb3-c98e9c6a3363 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:14.318325 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b27ad-a9b9-47f2-bbb3-c98e9c6a3363 admin admin] PATH_INFO: `/v3/projects/5138c47bd23f488f903788264ea9ff58/users/35c01178fd4d416387b30e1067bf430d/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:14.338615 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2b27ad-a9b9-47f2-bbb3-c98e9c6a3363 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5138c47bd23f488f903788264ea9ff58, user_id=35c01178fd4d416387b30e1067bf430d, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:14.341703 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2b27ad-a9b9-47f2-bbb3-c98e9c6a3363 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:14.355831 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 971/1947] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:52:14 2022] PUT /identity/v3/projects/5138c47bd23f488f903788264ea9ff58/users/35c01178fd4d416387b30e1067bf430d/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:14.366000 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b78c1-edcd-4e4d-9755-bcb6c8548e7b None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:14.366462 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b78c1-edcd-4e4d-9755-bcb6c8548e7b None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:14.366554 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b78c1-edcd-4e4d-9755-bcb6c8548e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:14.371045 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2b27ad-a9b9-47f2-bbb3-c98e9c6a3363 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:14.377290 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-6e2b27ad-a9b9-47f2-bbb3-c98e9c6a3363 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:14.408473 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-9b2b78c1-edcd-4e4d-9755-bcb6c8548e7b None None] MFA Rules not processed for user `35c01178fd4d416387b30e1067bf430d`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:14.410948 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-9b2b78c1-edcd-4e4d-9755-bcb6c8548e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:14.441419 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:14.451940 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2b27ad-a9b9-47f2-bbb3-c98e9c6a3363 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:14.457929 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0073e2c2-49dd-443b-bd9f-5a36531a17c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:14.458640 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073e2c2-49dd-443b-bd9f-5a36531a17c4 service cinder] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:14.459263 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073e2c2-49dd-443b-bd9f-5a36531a17c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:14.459598 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073e2c2-49dd-443b-bd9f-5a36531a17c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:14.461787 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-0073e2c2-49dd-443b-bd9f-5a36531a17c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:14.511779 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 977/1948] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:14 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:14.512391 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2b78c1-edcd-4e4d-9755-bcb6c8548e7b None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:14.513676 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-9b2b78c1-edcd-4e4d-9755-bcb6c8548e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:14.528522 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2b78c1-edcd-4e4d-9755-bcb6c8548e7b None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:14.532538 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea941b9d-0710-4ec9-96e1-cb0d7d09b0ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:14.533168 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea941b9d-0710-4ec9-96e1-cb0d7d09b0ac service cinder] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:14.533636 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea941b9d-0710-4ec9-96e1-cb0d7d09b0ac service cinder] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:14.533773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea941b9d-0710-4ec9-96e1-cb0d7d09b0ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:14.536302 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-ea941b9d-0710-4ec9-96e1-cb0d7d09b0ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:14.537230 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0073e2c2-49dd-443b-bd9f-5a36531a17c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:14.539102 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0073e2c2-49dd-443b-bd9f-5a36531a17c4 service cinder] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:14.549819 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 972/1949] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:52:14 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:52:14.592801 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0073e2c2-49dd-443b-bd9f-5a36531a17c4 service cinder] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:14.596258 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-0073e2c2-49dd-443b-bd9f-5a36531a17c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:14.612271 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea941b9d-0710-4ec9-96e1-cb0d7d09b0ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:14.614163 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea941b9d-0710-4ec9-96e1-cb0d7d09b0ac service cinder] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:14.628434 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 978/1950] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:52:14 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:52:14.704699 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0073e2c2-49dd-443b-bd9f-5a36531a17c4 service cinder] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:14.710165 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9d626-47ff-4dd3-b94c-c4e6045c31c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:14.710964 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9d626-47ff-4dd3-b94c-c4e6045c31c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:14.711254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9d626-47ff-4dd3-b94c-c4e6045c31c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:14.711528 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9d626-47ff-4dd3-b94c-c4e6045c31c2 admin admin] PATH_INFO: `/v3/projects/a0979ea73b614dcf9f120f5c713dd0fd` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:14.713087 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b9d626-47ff-4dd3-b94c-c4e6045c31c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0979ea73b614dcf9f120f5c713dd0fd)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:14.715208 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b9d626-47ff-4dd3-b94c-c4e6045c31c2 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:14.727355 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea941b9d-0710-4ec9-96e1-cb0d7d09b0ac service cinder] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:14.740523 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea941b9d-0710-4ec9-96e1-cb0d7d09b0ac service cinder] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:14.749935 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ddeff3-292f-45a8-a585-1a1e4b162a51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:14.750564 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ddeff3-292f-45a8-a585-1a1e4b162a51 service cinder] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:14.750865 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ddeff3-292f-45a8-a585-1a1e4b162a51 service cinder] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:14.751108 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ddeff3-292f-45a8-a585-1a1e4b162a51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:14.761692 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ddeff3-292f-45a8-a585-1a1e4b162a51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:14.763738 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ddeff3-292f-45a8-a585-1a1e4b162a51 service cinder] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:14.783168 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-74b9d626-47ff-4dd3-b94c-c4e6045c31c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0979ea73b614dcf9f120f5c713dd0fd', 'request_id': 'req-74b9d626-47ff-4dd3-b94c-c4e6045c31c2'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:14.784976 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 979/1951] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:52:14 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 12:52:14.791455 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-74b9d626-47ff-4dd3-b94c-c4e6045c31c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0979ea73b614dcf9f120f5c713dd0fd', 'request_id': 'req-74b9d626-47ff-4dd3-b94c-c4e6045c31c2'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:14.805484 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-74b9d626-47ff-4dd3-b94c-c4e6045c31c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0979ea73b614dcf9f120f5c713dd0fd', 'request_id': 'req-74b9d626-47ff-4dd3-b94c-c4e6045c31c2'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:14.831692 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-74b9d626-47ff-4dd3-b94c-c4e6045c31c2 admin admin] The token cache is being invalidate because project a0979ea73b614dcf9f120f5c713dd0fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:52:14.831997 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-74b9d626-47ff-4dd3-b94c-c4e6045c31c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:14.832847 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-74b9d626-47ff-4dd3-b94c-c4e6045c31c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:14.838281 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 973/1952] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:52:14 2022] DELETE /identity/v3/projects/a0979ea73b614dcf9f120f5c713dd0fd => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:52:14.839401 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ddeff3-292f-45a8-a585-1a1e4b162a51 service cinder] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:14.858199 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ddeff3-292f-45a8-a585-1a1e4b162a51 service cinder] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:14.982046 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:14.996233 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1fc4f4-e4dd-43e9-abce-03586395b9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:14.996949 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1fc4f4-e4dd-43e9-abce-03586395b9f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:14.998723 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1fc4f4-e4dd-43e9-abce-03586395b9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:14.999038 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1fc4f4-e4dd-43e9-abce-03586395b9f4 admin admin] PATH_INFO: `/v3/users/3fdbd72d7ad74c9bb0a8c81c3b921333` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:15.011706 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1fc4f4-e4dd-43e9-abce-03586395b9f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fdbd72d7ad74c9bb0a8c81c3b921333)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:15.011706 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1fc4f4-e4dd-43e9-abce-03586395b9f4 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:15.164571 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-9f1fc4f4-e4dd-43e9-abce-03586395b9f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fdbd72d7ad74c9bb0a8c81c3b921333'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:15.174366 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-9f1fc4f4-e4dd-43e9-abce-03586395b9f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fdbd72d7ad74c9bb0a8c81c3b921333'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:15.180573 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-9f1fc4f4-e4dd-43e9-abce-03586395b9f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fdbd72d7ad74c9bb0a8c81c3b921333'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:15.202000 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-9f1fc4f4-e4dd-43e9-abce-03586395b9f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fdbd72d7ad74c9bb0a8c81c3b921333'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:15.206229 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-9f1fc4f4-e4dd-43e9-abce-03586395b9f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fdbd72d7ad74c9bb0a8c81c3b921333'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:15.216073 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 980/1953] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:52:14 2022] DELETE /identity/v3/users/3fdbd72d7ad74c9bb0a8c81c3b921333 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:52:15.687515 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9d626-47ff-4dd3-b94c-c4e6045c31c2 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:15.689087 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-74b9d626-47ff-4dd3-b94c-c4e6045c31c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:15.703650 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9d626-47ff-4dd3-b94c-c4e6045c31c2 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:15.745779 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:15.796040 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f028ec45-e792-4b78-b959-324f72ab5b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:15.796774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f028ec45-e792-4b78-b959-324f72ab5b71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:15.797124 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f028ec45-e792-4b78-b959-324f72ab5b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:15.797836 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f028ec45-e792-4b78-b959-324f72ab5b71 admin admin] PATH_INFO: `/v3/projects/f6a99a83ae1743789d46512f0786c1c8` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:15.798776 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f028ec45-e792-4b78-b959-324f72ab5b71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6a99a83ae1743789d46512f0786c1c8)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:15.800844 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f028ec45-e792-4b78-b959-324f72ab5b71 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:15.855701 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-f028ec45-e792-4b78-b959-324f72ab5b71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6a99a83ae1743789d46512f0786c1c8', 'request_id': 'req-f028ec45-e792-4b78-b959-324f72ab5b71'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:15.862888 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-f028ec45-e792-4b78-b959-324f72ab5b71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6a99a83ae1743789d46512f0786c1c8', 'request_id': 'req-f028ec45-e792-4b78-b959-324f72ab5b71'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:15.871535 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-f028ec45-e792-4b78-b959-324f72ab5b71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6a99a83ae1743789d46512f0786c1c8', 'request_id': 'req-f028ec45-e792-4b78-b959-324f72ab5b71'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:15.893165 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-f028ec45-e792-4b78-b959-324f72ab5b71 admin admin] The token cache is being invalidate because project f6a99a83ae1743789d46512f0786c1c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:52:15.893601 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-f028ec45-e792-4b78-b959-324f72ab5b71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:15.894098 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-f028ec45-e792-4b78-b959-324f72ab5b71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:15.895885 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 974/1954] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:52:15 2022] DELETE /identity/v3/projects/f6a99a83ae1743789d46512f0786c1c8 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:15.901604 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceae5c3-eafd-466d-b62b-bd8f2a4ec7d3 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:15.902157 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceae5c3-eafd-466d-b62b-bd8f2a4ec7d3 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:15.902379 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceae5c3-eafd-466d-b62b-bd8f2a4ec7d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:15.936624 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-2ceae5c3-eafd-466d-b62b-bd8f2a4ec7d3 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:15.937869 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-2ceae5c3-eafd-466d-b62b-bd8f2a4ec7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:16.032472 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 981/1955] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:52:16.041072 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19765029-b32a-4eca-b0e3-db70c867efc3 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:16.041566 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19765029-b32a-4eca-b0e3-db70c867efc3 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:16.041985 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19765029-b32a-4eca-b0e3-db70c867efc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:16.073814 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-19765029-b32a-4eca-b0e3-db70c867efc3 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:16.075197 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-19765029-b32a-4eca-b0e3-db70c867efc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:16.087674 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 975/1956] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:16.092284 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceae5c3-eafd-466d-b62b-bd8f2a4ec7d3 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:16.103995 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceae5c3-eafd-466d-b62b-bd8f2a4ec7d3 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:16.109651 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecafcb5-2a51-49c2-af53-a96731ef4822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:16.110423 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecafcb5-2a51-49c2-af53-a96731ef4822 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:16.110838 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecafcb5-2a51-49c2-af53-a96731ef4822 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:16.111244 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecafcb5-2a51-49c2-af53-a96731ef4822 admin admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:16.112035 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecafcb5-2a51-49c2-af53-a96731ef4822 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:52:16.112562 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecafcb5-2a51-49c2-af53-a96731ef4822 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:16.114228 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecafcb5-2a51-49c2-af53-a96731ef4822 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:16.136343 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 982/1957] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:52:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:52:16.137221 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19765029-b32a-4eca-b0e3-db70c867efc3 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:16.146989 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19765029-b32a-4eca-b0e3-db70c867efc3 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:16.151653 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd8a54d-8e61-4710-9dea-3d8d89c3a043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:16.152305 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8a54d-8e61-4710-9dea-3d8d89c3a043 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:16.152584 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8a54d-8e61-4710-9dea-3d8d89c3a043 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:16.152856 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8a54d-8e61-4710-9dea-3d8d89c3a043 admin admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:16.153521 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd8a54d-8e61-4710-9dea-3d8d89c3a043 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:16.156232 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd8a54d-8e61-4710-9dea-3d8d89c3a043 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:16.222064 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 976/1958] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:52:16 2022] POST /identity/v3/projects => generated 363 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:52:16.230039 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecafcb5-2a51-49c2-af53-a96731ef4822 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:16.266575 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecafcb5-2a51-49c2-af53-a96731ef4822 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:16.414550 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d53ec2-2637-4124-9b78-cdee9430703b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:16.415308 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d53ec2-2637-4124-9b78-cdee9430703b admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:16.415724 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d53ec2-2637-4124-9b78-cdee9430703b admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:16.416091 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d53ec2-2637-4124-9b78-cdee9430703b admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:16.416881 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d53ec2-2637-4124-9b78-cdee9430703b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:16.419031 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d53ec2-2637-4124-9b78-cdee9430703b admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:16.432433 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:16.453756 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 983/1959] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:52:16 2022] POST /identity/v3/users => generated 282 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:52:16.455769 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd8a54d-8e61-4710-9dea-3d8d89c3a043 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:16.474833 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd8a54d-8e61-4710-9dea-3d8d89c3a043 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:16.482208 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df135d-413d-442a-a140-87fd22467198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:16.483385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df135d-413d-442a-a140-87fd22467198 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:16.483860 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df135d-413d-442a-a140-87fd22467198 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:16.484274 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df135d-413d-442a-a140-87fd22467198 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:16.486768 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4df135d-413d-442a-a140-87fd22467198 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:16.488903 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4df135d-413d-442a-a140-87fd22467198 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:16.510180 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 977/1960] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:16 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:52:16.514165 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d53ec2-2637-4124-9b78-cdee9430703b admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:16.530703 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d53ec2-2637-4124-9b78-cdee9430703b admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:16.543559 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309fb027-d4f4-4a7b-9701-5ecdb5f50a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:16.546442 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309fb027-d4f4-4a7b-9701-5ecdb5f50a8c admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:16.546857 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309fb027-d4f4-4a7b-9701-5ecdb5f50a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:16.547982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309fb027-d4f4-4a7b-9701-5ecdb5f50a8c admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:16.549322 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309fb027-d4f4-4a7b-9701-5ecdb5f50a8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:16.551868 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309fb027-d4f4-4a7b-9701-5ecdb5f50a8c admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:16.572710 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 984/1961] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:16 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 12:52:16.577244 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df135d-413d-442a-a140-87fd22467198 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:16.590376 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df135d-413d-442a-a140-87fd22467198 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:16.596488 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9915972f-78e9-43df-8d08-1f8aae2dc147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:16.597352 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9915972f-78e9-43df-8d08-1f8aae2dc147 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:16.597876 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9915972f-78e9-43df-8d08-1f8aae2dc147 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:16.598423 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9915972f-78e9-43df-8d08-1f8aae2dc147 admin admin] PATH_INFO: `/v3/projects/f023db081b754eccac273321ed119e50/users/433394a8270d4c7297ac838422f08327/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:16.623443 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9915972f-78e9-43df-8d08-1f8aae2dc147 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f023db081b754eccac273321ed119e50, user_id=433394a8270d4c7297ac838422f08327, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:16.627384 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9915972f-78e9-43df-8d08-1f8aae2dc147 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:16.646009 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 978/1962] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:52:16 2022] PUT /identity/v3/projects/f023db081b754eccac273321ed119e50/users/433394a8270d4c7297ac838422f08327/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:52:16.659247 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc81b7-1905-491e-a284-62961869418e None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:16.659705 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc81b7-1905-491e-a284-62961869418e None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:16.659993 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc81b7-1905-491e-a284-62961869418e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:16.706982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-4ccc81b7-1905-491e-a284-62961869418e None None] MFA Rules not processed for user `433394a8270d4c7297ac838422f08327`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:16.709657 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-4ccc81b7-1905-491e-a284-62961869418e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:16.824878 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 985/1963] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:16 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:16.830438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9915972f-78e9-43df-8d08-1f8aae2dc147 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:16.981703 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:16.989382 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9915972f-78e9-43df-8d08-1f8aae2dc147 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:17.004847 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9028428-a0f1-498d-b5ab-e16cba4b45ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:17.006319 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9028428-a0f1-498d-b5ab-e16cba4b45ab admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:17.006896 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9028428-a0f1-498d-b5ab-e16cba4b45ab admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:17.007965 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9028428-a0f1-498d-b5ab-e16cba4b45ab admin admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:17.009971 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9028428-a0f1-498d-b5ab-e16cba4b45ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:17.013969 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9028428-a0f1-498d-b5ab-e16cba4b45ab admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:17.071721 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 979/1964] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:52:16 2022] POST /identity/v3/projects => generated 363 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:52:17.073199 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc81b7-1905-491e-a284-62961869418e None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:17.092658 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc81b7-1905-491e-a284-62961869418e None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:17.170217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5145f090-2feb-43d3-a652-ea17f1e9cc2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:17.170994 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5145f090-2feb-43d3-a652-ea17f1e9cc2c admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:17.171377 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5145f090-2feb-43d3-a652-ea17f1e9cc2c admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:17.171808 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5145f090-2feb-43d3-a652-ea17f1e9cc2c admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:17.172591 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5145f090-2feb-43d3-a652-ea17f1e9cc2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:17.175046 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5145f090-2feb-43d3-a652-ea17f1e9cc2c admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:17.204613 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 986/1965] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:52:17 2022] POST /identity/v3/users => generated 288 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:52:17.210372 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9028428-a0f1-498d-b5ab-e16cba4b45ab admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:17.229025 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9028428-a0f1-498d-b5ab-e16cba4b45ab admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:17.236071 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f5660-3f92-4570-a6c6-b543b835ef38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:17.236896 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f5660-3f92-4570-a6c6-b543b835ef38 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:17.237277 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f5660-3f92-4570-a6c6-b543b835ef38 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:17.238359 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f5660-3f92-4570-a6c6-b543b835ef38 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:17.239179 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80f5660-3f92-4570-a6c6-b543b835ef38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:17.240972 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80f5660-3f92-4570-a6c6-b543b835ef38 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:17.255820 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 980/1966] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:17 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:52:17.258240 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5145f090-2feb-43d3-a652-ea17f1e9cc2c admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:17.268565 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5145f090-2feb-43d3-a652-ea17f1e9cc2c admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:17.275572 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8c6da-6035-4c07-b7cf-02c5a01dae92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:17.276527 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8c6da-6035-4c07-b7cf-02c5a01dae92 admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:17.276890 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8c6da-6035-4c07-b7cf-02c5a01dae92 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:17.277172 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8c6da-6035-4c07-b7cf-02c5a01dae92 admin admin] PATH_INFO: `/v3/projects/49c5dd7b61ac4bd2a51e6127e3f191c0/users/af1fec9c23a241ba98e8c74712c680a6/roles/0e78444e43c744668178971af4087ca0` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:17.294371 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa8c6da-6035-4c07-b7cf-02c5a01dae92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49c5dd7b61ac4bd2a51e6127e3f191c0, user_id=af1fec9c23a241ba98e8c74712c680a6, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:17.297156 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa8c6da-6035-4c07-b7cf-02c5a01dae92 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:17.313756 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 987/1967] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:52:17 2022] PUT /identity/v3/projects/49c5dd7b61ac4bd2a51e6127e3f191c0/users/af1fec9c23a241ba98e8c74712c680a6/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:52:17.317737 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f5660-3f92-4570-a6c6-b543b835ef38 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:17.328955 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f5660-3f92-4570-a6c6-b543b835ef38 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:17.414483 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9678e557-89d1-422c-8f3e-fb6c144dde05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:17.415230 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678e557-89d1-422c-8f3e-fb6c144dde05 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:17.415415 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678e557-89d1-422c-8f3e-fb6c144dde05 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:17.415767 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678e557-89d1-422c-8f3e-fb6c144dde05 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:17.417398 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9678e557-89d1-422c-8f3e-fb6c144dde05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:17.419237 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9678e557-89d1-422c-8f3e-fb6c144dde05 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:17.439343 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 981/1968] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:17 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:52:17.440421 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8c6da-6035-4c07-b7cf-02c5a01dae92 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:17.456085 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8c6da-6035-4c07-b7cf-02c5a01dae92 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:17.465428 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030d825a-1e83-4bbf-8827-a755dfb84ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:17.466840 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d825a-1e83-4bbf-8827-a755dfb84ce9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:17.467286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d825a-1e83-4bbf-8827-a755dfb84ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:17.467920 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d825a-1e83-4bbf-8827-a755dfb84ce9 admin admin] PATH_INFO: `/v3/projects/49c5dd7b61ac4bd2a51e6127e3f191c0/users/af1fec9c23a241ba98e8c74712c680a6/roles/0e78444e43c744668178971af4087ca0` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:17.471581 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030d825a-1e83-4bbf-8827-a755dfb84ce9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49c5dd7b61ac4bd2a51e6127e3f191c0, user_id=af1fec9c23a241ba98e8c74712c680a6, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:17.475439 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030d825a-1e83-4bbf-8827-a755dfb84ce9 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:17.536394 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6578a17d-32fe-4720-b1e0-72841e56a61a None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:17.536860 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6578a17d-32fe-4720-b1e0-72841e56a61a None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:17.537431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6578a17d-32fe-4720-b1e0-72841e56a61a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:17.539755 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 988/1969] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:52:17 2022] PUT /identity/v3/projects/49c5dd7b61ac4bd2a51e6127e3f191c0/users/af1fec9c23a241ba98e8c74712c680a6/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:17.579785 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-6578a17d-32fe-4720-b1e0-72841e56a61a None None] MFA Rules not processed for user `af1fec9c23a241ba98e8c74712c680a6`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:17.581814 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-6578a17d-32fe-4720-b1e0-72841e56a61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:17.712316 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 982/1970] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:17 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:17.769583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030d825a-1e83-4bbf-8827-a755dfb84ce9 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:17.772539 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-030d825a-1e83-4bbf-8827-a755dfb84ce9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:17.821222 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:17.871010 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030d825a-1e83-4bbf-8827-a755dfb84ce9 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:17.875662 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990ec561-1418-4cc3-bec0-9958f94dd3b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:17.876341 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990ec561-1418-4cc3-bec0-9958f94dd3b5 service cinder] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:17.876610 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990ec561-1418-4cc3-bec0-9958f94dd3b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:17.877070 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990ec561-1418-4cc3-bec0-9958f94dd3b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:17.891383 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990ec561-1418-4cc3-bec0-9958f94dd3b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:17.893011 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990ec561-1418-4cc3-bec0-9958f94dd3b5 service cinder] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:17.908071 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 989/1971] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:52:17 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:52:18.488921 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6578a17d-32fe-4720-b1e0-72841e56a61a None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:18.556752 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:18.573728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6578a17d-32fe-4720-b1e0-72841e56a61a None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:18.578800 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744dd49a-4019-4002-be61-bf71e9892ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:18.579470 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744dd49a-4019-4002-be61-bf71e9892ddd admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:18.579782 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744dd49a-4019-4002-be61-bf71e9892ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:18.580059 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744dd49a-4019-4002-be61-bf71e9892ddd admin admin] PATH_INFO: `/v3/users/433394a8270d4c7297ac838422f08327` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:18.581745 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744dd49a-4019-4002-be61-bf71e9892ddd admin admin] RBAC: Authorizing `identity:delete_user(user_id=433394a8270d4c7297ac838422f08327)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:18.585845 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744dd49a-4019-4002-be61-bf71e9892ddd admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:18.645294 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-744dd49a-4019-4002-be61-bf71e9892ddd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '433394a8270d4c7297ac838422f08327'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:18.645971 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-744dd49a-4019-4002-be61-bf71e9892ddd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '433394a8270d4c7297ac838422f08327'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:18.647189 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-744dd49a-4019-4002-be61-bf71e9892ddd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '433394a8270d4c7297ac838422f08327'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:18.665676 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-744dd49a-4019-4002-be61-bf71e9892ddd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '433394a8270d4c7297ac838422f08327'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:18.677253 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-744dd49a-4019-4002-be61-bf71e9892ddd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '433394a8270d4c7297ac838422f08327'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:18.694029 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 983/1972] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:52:18 2022] DELETE /identity/v3/users/433394a8270d4c7297ac838422f08327 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:18.698860 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf663f4-c8ee-443b-bfe6-308bcd6125ae None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:18.699750 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf663f4-c8ee-443b-bfe6-308bcd6125ae None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:18.700042 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf663f4-c8ee-443b-bfe6-308bcd6125ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:18.725744 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-caf663f4-c8ee-443b-bfe6-308bcd6125ae None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:18.727968 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-caf663f4-c8ee-443b-bfe6-308bcd6125ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:18.838005 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 990/1973] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:52:18.855132 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744dd49a-4019-4002-be61-bf71e9892ddd admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:18.857271 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-744dd49a-4019-4002-be61-bf71e9892ddd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:19.025404 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744dd49a-4019-4002-be61-bf71e9892ddd admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:19.036975 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f385e-a3e2-41de-bc59-2cad6bb997dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:19.037930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f385e-a3e2-41de-bc59-2cad6bb997dc service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:19.042350 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f385e-a3e2-41de-bc59-2cad6bb997dc service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:19.042639 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f385e-a3e2-41de-bc59-2cad6bb997dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:19.062850 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882f385e-a3e2-41de-bc59-2cad6bb997dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:19.065085 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882f385e-a3e2-41de-bc59-2cad6bb997dc service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:19.089474 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 984/1974] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:52:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:52:19.717173 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf663f4-c8ee-443b-bfe6-308bcd6125ae None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:19.718719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-caf663f4-c8ee-443b-bfe6-308bcd6125ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:19.730904 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf663f4-c8ee-443b-bfe6-308bcd6125ae None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:19.736671 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be25ed6a-5828-4d4c-8541-c8449985df8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:19.737561 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25ed6a-5828-4d4c-8541-c8449985df8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:19.738028 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25ed6a-5828-4d4c-8541-c8449985df8c admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:19.738305 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25ed6a-5828-4d4c-8541-c8449985df8c admin admin] PATH_INFO: `/v3/projects/f023db081b754eccac273321ed119e50` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:19.740839 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be25ed6a-5828-4d4c-8541-c8449985df8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f023db081b754eccac273321ed119e50)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:19.742947 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be25ed6a-5828-4d4c-8541-c8449985df8c admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:19.800097 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-be25ed6a-5828-4d4c-8541-c8449985df8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f023db081b754eccac273321ed119e50', 'request_id': 'req-be25ed6a-5828-4d4c-8541-c8449985df8c'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:19.809633 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-be25ed6a-5828-4d4c-8541-c8449985df8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f023db081b754eccac273321ed119e50', 'request_id': 'req-be25ed6a-5828-4d4c-8541-c8449985df8c'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:19.818670 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-be25ed6a-5828-4d4c-8541-c8449985df8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f023db081b754eccac273321ed119e50', 'request_id': 'req-be25ed6a-5828-4d4c-8541-c8449985df8c'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:19.833110 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-be25ed6a-5828-4d4c-8541-c8449985df8c admin admin] The token cache is being invalidate because project f023db081b754eccac273321ed119e50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:52:19.833581 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-be25ed6a-5828-4d4c-8541-c8449985df8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:19.834449 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-be25ed6a-5828-4d4c-8541-c8449985df8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:19.837198 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 991/1975] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:52:19 2022] DELETE /identity/v3/projects/f023db081b754eccac273321ed119e50 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:19.840943 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f385e-a3e2-41de-bc59-2cad6bb997dc service neutron] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:19.855220 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f385e-a3e2-41de-bc59-2cad6bb997dc service neutron] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:19.938190 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe0c41-f066-4ec5-9b42-b690a5238f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:19.939100 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe0c41-f066-4ec5-9b42-b690a5238f1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:19.939513 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe0c41-f066-4ec5-9b42-b690a5238f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:19.939896 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe0c41-f066-4ec5-9b42-b690a5238f1e admin admin] PATH_INFO: `/v3/users/af1fec9c23a241ba98e8c74712c680a6` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:19.941329 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fe0c41-f066-4ec5-9b42-b690a5238f1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=af1fec9c23a241ba98e8c74712c680a6)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:19.943477 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fe0c41-f066-4ec5-9b42-b690a5238f1e admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:19.960901 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:20.006912 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-57fe0c41-f066-4ec5-9b42-b690a5238f1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af1fec9c23a241ba98e8c74712c680a6'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:20.007602 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-57fe0c41-f066-4ec5-9b42-b690a5238f1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af1fec9c23a241ba98e8c74712c680a6'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:20.008331 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-57fe0c41-f066-4ec5-9b42-b690a5238f1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af1fec9c23a241ba98e8c74712c680a6'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:20.022798 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-57fe0c41-f066-4ec5-9b42-b690a5238f1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af1fec9c23a241ba98e8c74712c680a6'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:20.029866 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-57fe0c41-f066-4ec5-9b42-b690a5238f1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af1fec9c23a241ba98e8c74712c680a6'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:20.044808 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 985/1976] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:52:19 2022] DELETE /identity/v3/users/af1fec9c23a241ba98e8c74712c680a6 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:52:20.259126 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be25ed6a-5828-4d4c-8541-c8449985df8c admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:20.261435 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-be25ed6a-5828-4d4c-8541-c8449985df8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:20.278609 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be25ed6a-5828-4d4c-8541-c8449985df8c admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:20.374472 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe0c41-f066-4ec5-9b42-b690a5238f1e admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:20.376184 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-57fe0c41-f066-4ec5-9b42-b690a5238f1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:20.405510 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:20.426480 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1642f2ca-cb25-4d87-bc8d-f7d1359c9620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:20.428173 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642f2ca-cb25-4d87-bc8d-f7d1359c9620 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:20.428649 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642f2ca-cb25-4d87-bc8d-f7d1359c9620 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:20.429211 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642f2ca-cb25-4d87-bc8d-f7d1359c9620 admin admin] PATH_INFO: `/v3/users/ff646cd99262476fa5a06f490dc4107f` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:20.431079 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1642f2ca-cb25-4d87-bc8d-f7d1359c9620 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff646cd99262476fa5a06f490dc4107f)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:20.434307 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1642f2ca-cb25-4d87-bc8d-f7d1359c9620 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:20.499168 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe0c41-f066-4ec5-9b42-b690a5238f1e admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:20.504806 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574cccc0-dc2d-4c3a-9349-cef82680ca93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:20.505476 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574cccc0-dc2d-4c3a-9349-cef82680ca93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:20.506887 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574cccc0-dc2d-4c3a-9349-cef82680ca93 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:20.506887 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574cccc0-dc2d-4c3a-9349-cef82680ca93 admin admin] PATH_INFO: `/v3/projects/49c5dd7b61ac4bd2a51e6127e3f191c0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:20.507998 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574cccc0-dc2d-4c3a-9349-cef82680ca93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49c5dd7b61ac4bd2a51e6127e3f191c0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:20.510155 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574cccc0-dc2d-4c3a-9349-cef82680ca93 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:20.528134 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-1642f2ca-cb25-4d87-bc8d-f7d1359c9620 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff646cd99262476fa5a06f490dc4107f'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:20.539760 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-1642f2ca-cb25-4d87-bc8d-f7d1359c9620 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff646cd99262476fa5a06f490dc4107f'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:20.540658 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-1642f2ca-cb25-4d87-bc8d-f7d1359c9620 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff646cd99262476fa5a06f490dc4107f'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:20.550038 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-1642f2ca-cb25-4d87-bc8d-f7d1359c9620 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff646cd99262476fa5a06f490dc4107f'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:20.571265 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-1642f2ca-cb25-4d87-bc8d-f7d1359c9620 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff646cd99262476fa5a06f490dc4107f'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:20.574154 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-574cccc0-dc2d-4c3a-9349-cef82680ca93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49c5dd7b61ac4bd2a51e6127e3f191c0', 'request_id': 'req-574cccc0-dc2d-4c3a-9349-cef82680ca93'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:20.582294 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-574cccc0-dc2d-4c3a-9349-cef82680ca93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49c5dd7b61ac4bd2a51e6127e3f191c0', 'request_id': 'req-574cccc0-dc2d-4c3a-9349-cef82680ca93'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:20.590913 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 992/1977] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:52:20 2022] DELETE /identity/v3/users/ff646cd99262476fa5a06f490dc4107f => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:20.592579 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-574cccc0-dc2d-4c3a-9349-cef82680ca93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49c5dd7b61ac4bd2a51e6127e3f191c0', 'request_id': 'req-574cccc0-dc2d-4c3a-9349-cef82680ca93'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:20.593290 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27d191-a31e-43be-b922-269266ffc093 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:20.593597 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27d191-a31e-43be-b922-269266ffc093 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:20.593874 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27d191-a31e-43be-b922-269266ffc093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:20.608987 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-574cccc0-dc2d-4c3a-9349-cef82680ca93 admin admin] The token cache is being invalidate because project 49c5dd7b61ac4bd2a51e6127e3f191c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:52:20.616306 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-574cccc0-dc2d-4c3a-9349-cef82680ca93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:20.622761 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-574cccc0-dc2d-4c3a-9349-cef82680ca93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:20.625604 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 986/1978] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:52:20 2022] DELETE /identity/v3/projects/49c5dd7b61ac4bd2a51e6127e3f191c0 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:52:20.630208 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453c38b-9ce3-4b22-ae69-9b6ff459f67b None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:20.631211 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453c38b-9ce3-4b22-ae69-9b6ff459f67b None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:20.631494 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453c38b-9ce3-4b22-ae69-9b6ff459f67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:20.633906 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-7b27d191-a31e-43be-b922-269266ffc093 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:20.635125 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-7b27d191-a31e-43be-b922-269266ffc093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:20.657160 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-3453c38b-9ce3-4b22-ae69-9b6ff459f67b None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:20.658103 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-3453c38b-9ce3-4b22-ae69-9b6ff459f67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:20.681315 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:20.730613 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 993/1979] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:52:20.739793 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 987/1980] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:52:20.742434 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3453c38b-9ce3-4b22-ae69-9b6ff459f67b None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:20.742941 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d66f37-dd70-4d3a-8c24-6ad7109b6ace None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:20.743234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d66f37-dd70-4d3a-8c24-6ad7109b6ace None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:20.743512 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d66f37-dd70-4d3a-8c24-6ad7109b6ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:20.744160 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-3453c38b-9ce3-4b22-ae69-9b6ff459f67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:20.776255 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-42d66f37-dd70-4d3a-8c24-6ad7109b6ace None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:20.777538 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-42d66f37-dd70-4d3a-8c24-6ad7109b6ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:20.794024 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 994/1981] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:20.795488 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d66f37-dd70-4d3a-8c24-6ad7109b6ace None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:20.805278 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d66f37-dd70-4d3a-8c24-6ad7109b6ace None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:20.811215 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccdc277-1863-4dfe-b449-952a03cb7bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:20.811871 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdc277-1863-4dfe-b449-952a03cb7bac admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:20.812153 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdc277-1863-4dfe-b449-952a03cb7bac admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:20.812419 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdc277-1863-4dfe-b449-952a03cb7bac admin admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:20.813318 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccdc277-1863-4dfe-b449-952a03cb7bac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:52:20.813796 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccdc277-1863-4dfe-b449-952a03cb7bac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:20.815619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccdc277-1863-4dfe-b449-952a03cb7bac admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:20.817613 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3453c38b-9ce3-4b22-ae69-9b6ff459f67b None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:20.822531 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ffa90-9a9f-471a-93c8-dc9c4712ac79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:20.823151 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ffa90-9a9f-471a-93c8-dc9c4712ac79 service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:20.823426 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ffa90-9a9f-471a-93c8-dc9c4712ac79 service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:20.823855 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ffa90-9a9f-471a-93c8-dc9c4712ac79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:20.837576 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 995/1982] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:52:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:52:20.837759 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186ffa90-9a9f-471a-93c8-dc9c4712ac79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:20.839125 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186ffa90-9a9f-471a-93c8-dc9c4712ac79 service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:20.841588 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccdc277-1863-4dfe-b449-952a03cb7bac admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:20.852554 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccdc277-1863-4dfe-b449-952a03cb7bac admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:20.858444 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb31e61-dbea-45d0-a80e-1816ab0d4919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:20.859281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb31e61-dbea-45d0-a80e-1816ab0d4919 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:20.859883 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb31e61-dbea-45d0-a80e-1816ab0d4919 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:20.860272 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb31e61-dbea-45d0-a80e-1816ab0d4919 admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:20.862055 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb31e61-dbea-45d0-a80e-1816ab0d4919 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:20.864649 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb31e61-dbea-45d0-a80e-1816ab0d4919 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:20.867469 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 988/1983] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:52:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:52:20.908090 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 996/1984] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:52:20 2022] POST /identity/v3/projects => generated 383 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:52:20.909443 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ffa90-9a9f-471a-93c8-dc9c4712ac79 service neutron] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:20.923710 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ffa90-9a9f-471a-93c8-dc9c4712ac79 service neutron] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:21.013401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7c410-0c73-4069-90e4-9d77374dbcab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:21.014104 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7c410-0c73-4069-90e4-9d77374dbcab admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:21.020755 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7c410-0c73-4069-90e4-9d77374dbcab admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:21.020755 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7c410-0c73-4069-90e4-9d77374dbcab admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:21.022079 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b7c410-0c73-4069-90e4-9d77374dbcab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:21.024170 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b7c410-0c73-4069-90e4-9d77374dbcab admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:21.067359 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb31e61-dbea-45d0-a80e-1816ab0d4919 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:21.067692 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 989/1985] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:52:20 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:52:21.083817 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb31e61-dbea-45d0-a80e-1816ab0d4919 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:21.108977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c7564-4ba5-4af7-887d-86e072c4b0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:21.109634 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c7564-4ba5-4af7-887d-86e072c4b0a4 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:21.109915 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c7564-4ba5-4af7-887d-86e072c4b0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:21.110215 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c7564-4ba5-4af7-887d-86e072c4b0a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:21.110928 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016c7564-4ba5-4af7-887d-86e072c4b0a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:21.112572 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016c7564-4ba5-4af7-887d-86e072c4b0a4 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:21.138077 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 997/1986] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:21 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:52:21.142576 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7c410-0c73-4069-90e4-9d77374dbcab admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:21.158568 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7c410-0c73-4069-90e4-9d77374dbcab admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:21.180910 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d8a882-15ea-44e4-bab0-d188753c4dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:21.181196 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8a882-15ea-44e4-bab0-d188753c4dde admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:21.181447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8a882-15ea-44e4-bab0-d188753c4dde admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:21.181722 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8a882-15ea-44e4-bab0-d188753c4dde admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:21.182449 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d8a882-15ea-44e4-bab0-d188753c4dde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:21.184115 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d8a882-15ea-44e4-bab0-d188753c4dde admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:21.218354 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 990/1987] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:21 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:52:21.222552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c7564-4ba5-4af7-887d-86e072c4b0a4 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:21.234229 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c7564-4ba5-4af7-887d-86e072c4b0a4 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:21.241548 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ea4f76-1e3b-4ca1-9f3b-cd1f9707371e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:21.242863 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea4f76-1e3b-4ca1-9f3b-cd1f9707371e admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:21.243611 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea4f76-1e3b-4ca1-9f3b-cd1f9707371e admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:21.244131 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea4f76-1e3b-4ca1-9f3b-cd1f9707371e admin admin] PATH_INFO: `/v3/projects/36edefec489849368c6b2849d86fbd36/users/ccf959c2ccd64ae19f2f4c6134d078af/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:21.269691 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:21.278996 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ea4f76-1e3b-4ca1-9f3b-cd1f9707371e admin admin] RBAC: Authorizing `identity:create_grant(project_id=36edefec489849368c6b2849d86fbd36, user_id=ccf959c2ccd64ae19f2f4c6134d078af, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:21.286317 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ea4f76-1e3b-4ca1-9f3b-cd1f9707371e admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:21.306517 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d8a882-15ea-44e4-bab0-d188753c4dde admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:21.308517 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-60d8a882-15ea-44e4-bab0-d188753c4dde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:21.331421 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d8a882-15ea-44e4-bab0-d188753c4dde admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:21.335413 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 998/1988] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:52:21 2022] PUT /identity/v3/projects/36edefec489849368c6b2849d86fbd36/users/ccf959c2ccd64ae19f2f4c6134d078af/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:21.337135 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1857136-fbc8-4b92-be52-ed8f916d67d4 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:21.338185 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1857136-fbc8-4b92-be52-ed8f916d67d4 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:21.338500 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1857136-fbc8-4b92-be52-ed8f916d67d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:21.375104 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-e1857136-fbc8-4b92-be52-ed8f916d67d4 None None] MFA Rules not processed for user `ccf959c2ccd64ae19f2f4c6134d078af`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:21.376645 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-e1857136-fbc8-4b92-be52-ed8f916d67d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:21.400447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:21.421368 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7bd4ee-f6ea-4533-a132-15ed3fc2671e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:21.422085 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bd4ee-f6ea-4533-a132-15ed3fc2671e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:21.422363 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bd4ee-f6ea-4533-a132-15ed3fc2671e admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:21.422635 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bd4ee-f6ea-4533-a132-15ed3fc2671e admin admin] PATH_INFO: `/v3/projects/e79476d97a854574a7de69c947ffe470` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:21.424026 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7bd4ee-f6ea-4533-a132-15ed3fc2671e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e79476d97a854574a7de69c947ffe470)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:21.438406 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7bd4ee-f6ea-4533-a132-15ed3fc2671e admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:21.493006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 999/1989] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:21.498569 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1857136-fbc8-4b92-be52-ed8f916d67d4 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:21.512804 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1857136-fbc8-4b92-be52-ed8f916d67d4 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:21.528115 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-3c7bd4ee-f6ea-4533-a132-15ed3fc2671e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e79476d97a854574a7de69c947ffe470', 'request_id': 'req-3c7bd4ee-f6ea-4533-a132-15ed3fc2671e'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:21.540361 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-3c7bd4ee-f6ea-4533-a132-15ed3fc2671e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e79476d97a854574a7de69c947ffe470', 'request_id': 'req-3c7bd4ee-f6ea-4533-a132-15ed3fc2671e'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:21.562383 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-3c7bd4ee-f6ea-4533-a132-15ed3fc2671e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e79476d97a854574a7de69c947ffe470', 'request_id': 'req-3c7bd4ee-f6ea-4533-a132-15ed3fc2671e'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:21.579994 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-3c7bd4ee-f6ea-4533-a132-15ed3fc2671e admin admin] The token cache is being invalidate because project e79476d97a854574a7de69c947ffe470 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:52:21.580447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-3c7bd4ee-f6ea-4533-a132-15ed3fc2671e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:21.581248 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-3c7bd4ee-f6ea-4533-a132-15ed3fc2671e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:21.583550 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 991/1990] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:52:21 2022] DELETE /identity/v3/projects/e79476d97a854574a7de69c947ffe470 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:21.587073 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7bd4ee-f6ea-4533-a132-15ed3fc2671e admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:21.589256 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-3c7bd4ee-f6ea-4533-a132-15ed3fc2671e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:21.602791 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d3650a-3ddc-498d-aeb2-f1770ef9e3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:21.603905 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3650a-3ddc-498d-aeb2-f1770ef9e3f4 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:21.604458 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3650a-3ddc-498d-aeb2-f1770ef9e3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:21.604741 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3650a-3ddc-498d-aeb2-f1770ef9e3f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:21.605583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d3650a-3ddc-498d-aeb2-f1770ef9e3f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:21.609181 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d3650a-3ddc-498d-aeb2-f1770ef9e3f4 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:21.662895 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1000/1991] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:52:21 2022] POST /identity/v3/projects => generated 381 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:52:21.666617 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d3650a-3ddc-498d-aeb2-f1770ef9e3f4 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:21.670348 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-e4d3650a-3ddc-498d-aeb2-f1770ef9e3f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:21.718735 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7bd4ee-f6ea-4533-a132-15ed3fc2671e admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:21.808896 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b176d2e9-3d19-461d-8310-ffd04148d470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:21.809713 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b176d2e9-3d19-461d-8310-ffd04148d470 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:21.810228 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b176d2e9-3d19-461d-8310-ffd04148d470 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:21.810642 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b176d2e9-3d19-461d-8310-ffd04148d470 admin admin] PATH_INFO: `/v3/users/2dc8b56998f944398ea495bd1b276ad9` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:21.819480 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d3650a-3ddc-498d-aeb2-f1770ef9e3f4 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:21.823704 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b176d2e9-3d19-461d-8310-ffd04148d470 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dc8b56998f944398ea495bd1b276ad9)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:21.825789 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b176d2e9-3d19-461d-8310-ffd04148d470 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:21.827058 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c917a4e-801a-4130-bf86-d61a420e4d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:21.827903 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c917a4e-801a-4130-bf86-d61a420e4d24 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:21.828183 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c917a4e-801a-4130-bf86-d61a420e4d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:21.828513 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c917a4e-801a-4130-bf86-d61a420e4d24 admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:21.829337 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c917a4e-801a-4130-bf86-d61a420e4d24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:21.831344 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c917a4e-801a-4130-bf86-d61a420e4d24 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:21.848912 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:21.881384 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:21.883085 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1001/1992] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:52:21 2022] POST /identity/v3/users => generated 297 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:52:21.890963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c917a4e-801a-4130-bf86-d61a420e4d24 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:21.916313 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c917a4e-801a-4130-bf86-d61a420e4d24 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:21.920131 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-b176d2e9-3d19-461d-8310-ffd04148d470 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dc8b56998f944398ea495bd1b276ad9'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:21.920864 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-b176d2e9-3d19-461d-8310-ffd04148d470 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dc8b56998f944398ea495bd1b276ad9'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:21.921532 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-b176d2e9-3d19-461d-8310-ffd04148d470 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dc8b56998f944398ea495bd1b276ad9'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:21.927688 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335e445f-6f94-4d89-91db-3b0b6e164c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:21.929892 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e445f-6f94-4d89-91db-3b0b6e164c5a admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:21.930288 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e445f-6f94-4d89-91db-3b0b6e164c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:21.931341 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e445f-6f94-4d89-91db-3b0b6e164c5a admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:21.932215 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335e445f-6f94-4d89-91db-3b0b6e164c5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:21.935628 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335e445f-6f94-4d89-91db-3b0b6e164c5a admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:21.943595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-b176d2e9-3d19-461d-8310-ffd04148d470 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dc8b56998f944398ea495bd1b276ad9'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:21.952620 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1002/1993] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:21 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:52:21.953993 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-b176d2e9-3d19-461d-8310-ffd04148d470 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dc8b56998f944398ea495bd1b276ad9'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:21.957738 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335e445f-6f94-4d89-91db-3b0b6e164c5a admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:21.960923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-335e445f-6f94-4d89-91db-3b0b6e164c5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:21.976313 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335e445f-6f94-4d89-91db-3b0b6e164c5a admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:21.980348 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 992/1994] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:52:21 2022] DELETE /identity/v3/users/2dc8b56998f944398ea495bd1b276ad9 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:52:22.064346 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8dadbf-4586-4b82-a487-0fcf2873401d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:22.065146 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8dadbf-4586-4b82-a487-0fcf2873401d admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:22.065497 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8dadbf-4586-4b82-a487-0fcf2873401d admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:22.065835 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8dadbf-4586-4b82-a487-0fcf2873401d admin admin] PATH_INFO: `/v3/projects/834f850d7aba40cabb76a6d79eb649de/users/854e32c1babc4608b5af4be1c026f6bb/roles/0e78444e43c744668178971af4087ca0` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:22.083886 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8dadbf-4586-4b82-a487-0fcf2873401d admin admin] RBAC: Authorizing `identity:create_grant(project_id=834f850d7aba40cabb76a6d79eb649de, user_id=854e32c1babc4608b5af4be1c026f6bb, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:22.087042 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8dadbf-4586-4b82-a487-0fcf2873401d admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:22.103221 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1003/1995] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:52:21 2022] PUT /identity/v3/projects/834f850d7aba40cabb76a6d79eb649de/users/854e32c1babc4608b5af4be1c026f6bb/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:52:22.107527 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b176d2e9-3d19-461d-8310-ffd04148d470 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:22.119016 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b176d2e9-3d19-461d-8310-ffd04148d470 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:22.197953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1365ee9-722c-4cf1-91d8-a8172688c248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:22.198760 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1365ee9-722c-4cf1-91d8-a8172688c248 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:22.199204 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1365ee9-722c-4cf1-91d8-a8172688c248 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:22.199488 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1365ee9-722c-4cf1-91d8-a8172688c248 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:22.200220 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1365ee9-722c-4cf1-91d8-a8172688c248 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:22.202643 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1365ee9-722c-4cf1-91d8-a8172688c248 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:22.213686 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 993/1996] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:52:22.217831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8dadbf-4586-4b82-a487-0fcf2873401d admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:22.229195 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8dadbf-4586-4b82-a487-0fcf2873401d admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:22.234293 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b5408-83bd-4a70-bfe9-424324a0c905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:22.234980 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b5408-83bd-4a70-bfe9-424324a0c905 admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:22.235258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b5408-83bd-4a70-bfe9-424324a0c905 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:22.235567 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b5408-83bd-4a70-bfe9-424324a0c905 admin admin] PATH_INFO: `/v3/projects/834f850d7aba40cabb76a6d79eb649de/users/854e32c1babc4608b5af4be1c026f6bb/roles/0e78444e43c744668178971af4087ca0` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:22.237581 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69b5408-83bd-4a70-bfe9-424324a0c905 admin admin] RBAC: Authorizing `identity:create_grant(project_id=834f850d7aba40cabb76a6d79eb649de, user_id=854e32c1babc4608b5af4be1c026f6bb, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:22.240894 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69b5408-83bd-4a70-bfe9-424324a0c905 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:22.253173 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1004/1997] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:52:22 2022] PUT /identity/v3/projects/834f850d7aba40cabb76a6d79eb649de/users/854e32c1babc4608b5af4be1c026f6bb/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:22.263945 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76c585-cc03-4bf7-98cb-9be9a117bbe0 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:22.264236 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76c585-cc03-4bf7-98cb-9be9a117bbe0 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:22.264517 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76c585-cc03-4bf7-98cb-9be9a117bbe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:22.296620 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b5408-83bd-4a70-bfe9-424324a0c905 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:22.298809 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-b69b5408-83bd-4a70-bfe9-424324a0c905 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:22.299148 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-3b76c585-cc03-4bf7-98cb-9be9a117bbe0 None None] MFA Rules not processed for user `854e32c1babc4608b5af4be1c026f6bb`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:22.300586 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-3b76c585-cc03-4bf7-98cb-9be9a117bbe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:22.372784 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b5408-83bd-4a70-bfe9-424324a0c905 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:22.379671 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838b041-75cd-4502-8562-3111771a2f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:22.380573 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838b041-75cd-4502-8562-3111771a2f50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:22.380854 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838b041-75cd-4502-8562-3111771a2f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:22.381244 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838b041-75cd-4502-8562-3111771a2f50 admin admin] PATH_INFO: `/v3/projects/a227bb3ec80e47dbb1a11e4bf00251e9` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:22.382417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3838b041-75cd-4502-8562-3111771a2f50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a227bb3ec80e47dbb1a11e4bf00251e9)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:22.383571 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 994/1998] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:22 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:22.384401 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3838b041-75cd-4502-8562-3111771a2f50 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:22.396701 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b76c585-cc03-4bf7-98cb-9be9a117bbe0 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:22.398116 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-3b76c585-cc03-4bf7-98cb-9be9a117bbe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:22.430523 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:22.446163 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-3838b041-75cd-4502-8562-3111771a2f50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a227bb3ec80e47dbb1a11e4bf00251e9', 'request_id': 'req-3838b041-75cd-4502-8562-3111771a2f50'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:22.453715 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-3838b041-75cd-4502-8562-3111771a2f50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a227bb3ec80e47dbb1a11e4bf00251e9', 'request_id': 'req-3838b041-75cd-4502-8562-3111771a2f50'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:22.457929 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:22.462544 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-3838b041-75cd-4502-8562-3111771a2f50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a227bb3ec80e47dbb1a11e4bf00251e9', 'request_id': 'req-3838b041-75cd-4502-8562-3111771a2f50'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:22.476209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-3838b041-75cd-4502-8562-3111771a2f50 admin admin] The token cache is being invalidate because project a227bb3ec80e47dbb1a11e4bf00251e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:52:22.476499 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-3838b041-75cd-4502-8562-3111771a2f50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:22.477168 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-3838b041-75cd-4502-8562-3111771a2f50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:22.478668 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b76c585-cc03-4bf7-98cb-9be9a117bbe0 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:22.483290 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1005/1999] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:52:22 2022] DELETE /identity/v3/projects/a227bb3ec80e47dbb1a11e4bf00251e9 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:52:22.484781 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71551f-a434-48f1-a34d-653fd3726dde None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:22.485189 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71551f-a434-48f1-a34d-653fd3726dde None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:22.485343 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71551f-a434-48f1-a34d-653fd3726dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:22.506516 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-7b71551f-a434-48f1-a34d-653fd3726dde None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:22.507883 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-7b71551f-a434-48f1-a34d-653fd3726dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:22.537433 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc4849-0976-4a34-8b00-175631085208 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:22.539107 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc4849-0976-4a34-8b00-175631085208 service cinder] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:22.539481 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc4849-0976-4a34-8b00-175631085208 service cinder] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:22.539743 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc4849-0976-4a34-8b00-175631085208 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:22.541997 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-74dc4849-0976-4a34-8b00-175631085208 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:22.590439 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1006/2000] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:22.592559 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8997537-d217-4172-b4d8-6a5f251fa31b None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:22.592921 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8997537-d217-4172-b4d8-6a5f251fa31b None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:22.593270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8997537-d217-4172-b4d8-6a5f251fa31b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:22.619757 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-d8997537-d217-4172-b4d8-6a5f251fa31b None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:22.620983 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-d8997537-d217-4172-b4d8-6a5f251fa31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:22.624010 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc4849-0976-4a34-8b00-175631085208 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:22.625489 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc4849-0976-4a34-8b00-175631085208 service cinder] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:22.630069 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1007/2001] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:22.635328 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8997537-d217-4172-b4d8-6a5f251fa31b None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:22.641342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 995/2002] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:52:22 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 12:52:22.651055 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8997537-d217-4172-b4d8-6a5f251fa31b None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:22.656927 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd5ccc-7949-4235-a897-d0b59af057df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:22.657658 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd5ccc-7949-4235-a897-d0b59af057df admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:22.657941 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd5ccc-7949-4235-a897-d0b59af057df admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:22.658227 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd5ccc-7949-4235-a897-d0b59af057df admin admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:22.658916 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd5ccc-7949-4235-a897-d0b59af057df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:52:22.659214 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd5ccc-7949-4235-a897-d0b59af057df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:22.661519 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd5ccc-7949-4235-a897-d0b59af057df admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:22.680141 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1008/2003] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:52:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:52:22.684710 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc4849-0976-4a34-8b00-175631085208 service cinder] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:22.698862 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc4849-0976-4a34-8b00-175631085208 service cinder] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:22.717459 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c7b05-d7ad-4647-8503-a6de9a428f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:22.718141 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c7b05-d7ad-4647-8503-a6de9a428f0e admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:22.718425 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c7b05-d7ad-4647-8503-a6de9a428f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:22.718702 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c7b05-d7ad-4647-8503-a6de9a428f0e admin admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:22.719385 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c7b05-d7ad-4647-8503-a6de9a428f0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:22.721377 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c7b05-d7ad-4647-8503-a6de9a428f0e admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:22.770990 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 996/2004] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:52:22 2022] POST /identity/v3/projects => generated 375 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:52:22.771610 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd5ccc-7949-4235-a897-d0b59af057df admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:22.778214 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd5ccc-7949-4235-a897-d0b59af057df admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:22.804216 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c7b05-d7ad-4647-8503-a6de9a428f0e admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:22.805917 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-f52c7b05-d7ad-4647-8503-a6de9a428f0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:22.841665 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab070393-1b68-45b7-b414-7777c1f398b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:22.842385 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab070393-1b68-45b7-b414-7777c1f398b4 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:22.842673 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab070393-1b68-45b7-b414-7777c1f398b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:22.842993 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab070393-1b68-45b7-b414-7777c1f398b4 admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:22.843623 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab070393-1b68-45b7-b414-7777c1f398b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:22.846266 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab070393-1b68-45b7-b414-7777c1f398b4 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:22.867410 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c7b05-d7ad-4647-8503-a6de9a428f0e admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:22.870207 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1009/2005] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:52:22 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:52:22.871166 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b7aef-8e53-431f-8991-b5339261cad5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:22.871818 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b7aef-8e53-431f-8991-b5339261cad5 service cinder] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:22.872101 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b7aef-8e53-431f-8991-b5339261cad5 service cinder] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:22.872374 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b7aef-8e53-431f-8991-b5339261cad5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:22.874463 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-be6b7aef-8e53-431f-8991-b5339261cad5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:22.874583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab070393-1b68-45b7-b414-7777c1f398b4 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:22.885114 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab070393-1b68-45b7-b414-7777c1f398b4 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:22.889942 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b07305-e7b4-4378-b2f5-7c7ddea9bdf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:22.890474 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b07305-e7b4-4378-b2f5-7c7ddea9bdf3 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:22.890758 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b07305-e7b4-4378-b2f5-7c7ddea9bdf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:22.891009 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b07305-e7b4-4378-b2f5-7c7ddea9bdf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:22.891735 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b07305-e7b4-4378-b2f5-7c7ddea9bdf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:22.893357 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b07305-e7b4-4378-b2f5-7c7ddea9bdf3 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:22.896278 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:22.910029 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1010/2006] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 12:52:22.914410 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b07305-e7b4-4378-b2f5-7c7ddea9bdf3 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:22.930050 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b07305-e7b4-4378-b2f5-7c7ddea9bdf3 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:22.935815 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788e75a-cfb0-4df1-bbae-1aa5e4a84a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:22.936517 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788e75a-cfb0-4df1-bbae-1aa5e4a84a68 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:22.936881 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788e75a-cfb0-4df1-bbae-1aa5e4a84a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:22.937221 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788e75a-cfb0-4df1-bbae-1aa5e4a84a68 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:22.938003 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d788e75a-cfb0-4df1-bbae-1aa5e4a84a68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:22.940371 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d788e75a-cfb0-4df1-bbae-1aa5e4a84a68 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:22.950924 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1011/2007] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 12:52:22.954911 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788e75a-cfb0-4df1-bbae-1aa5e4a84a68 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:22.956555 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6b7aef-8e53-431f-8991-b5339261cad5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:22.960049 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6b7aef-8e53-431f-8991-b5339261cad5 service cinder] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:22.972291 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 997/2008] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:52:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 12:52:22.982160 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788e75a-cfb0-4df1-bbae-1aa5e4a84a68 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:22.988228 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9884acf-8da9-45d9-8e62-24cae87192f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:22.989026 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9884acf-8da9-45d9-8e62-24cae87192f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:22.989380 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9884acf-8da9-45d9-8e62-24cae87192f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:22.989721 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9884acf-8da9-45d9-8e62-24cae87192f4 admin admin] PATH_INFO: `/v3/projects/ed3db84b8f2f4091875947bb77911b6f/users/eea3bddaa92f48aa867e0f97f99a990a/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:23.009939 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9884acf-8da9-45d9-8e62-24cae87192f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed3db84b8f2f4091875947bb77911b6f, user_id=eea3bddaa92f48aa867e0f97f99a990a, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:23.012878 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9884acf-8da9-45d9-8e62-24cae87192f4 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:23.029158 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1012/2009] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:52:22 2022] PUT /identity/v3/projects/ed3db84b8f2f4091875947bb77911b6f/users/eea3bddaa92f48aa867e0f97f99a990a/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:23.043991 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1675de0-08d9-480e-822f-483fb996ae11 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:23.044377 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1675de0-08d9-480e-822f-483fb996ae11 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:23.045431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1675de0-08d9-480e-822f-483fb996ae11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:23.087798 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-f1675de0-08d9-480e-822f-483fb996ae11 None None] MFA Rules not processed for user `eea3bddaa92f48aa867e0f97f99a990a`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:23.091719 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-f1675de0-08d9-480e-822f-483fb996ae11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:23.175570 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 998/2010] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:23.180530 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9884acf-8da9-45d9-8e62-24cae87192f4 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:23.194633 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:23.271625 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9884acf-8da9-45d9-8e62-24cae87192f4 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:23.282758 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b9e4a-8efe-4fb8-8d2e-e5f5fd92e48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:23.284138 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b9e4a-8efe-4fb8-8d2e-e5f5fd92e48d admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:23.284853 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b9e4a-8efe-4fb8-8d2e-e5f5fd92e48d admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:23.285358 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b9e4a-8efe-4fb8-8d2e-e5f5fd92e48d admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:23.288245 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982b9e4a-8efe-4fb8-8d2e-e5f5fd92e48d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:23.291343 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982b9e4a-8efe-4fb8-8d2e-e5f5fd92e48d admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:23.333523 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1013/2011] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:52:23 2022] POST /identity/v3/projects => generated 377 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:52:23.342256 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1675de0-08d9-480e-822f-483fb996ae11 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:23.368073 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1675de0-08d9-480e-822f-483fb996ae11 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:23.449461 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e162eb2b-36a4-4514-b7df-e13822b564d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:23.450080 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e162eb2b-36a4-4514-b7df-e13822b564d1 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:23.450359 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e162eb2b-36a4-4514-b7df-e13822b564d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:23.450628 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e162eb2b-36a4-4514-b7df-e13822b564d1 admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:23.451292 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e162eb2b-36a4-4514-b7df-e13822b564d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:23.453511 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e162eb2b-36a4-4514-b7df-e13822b564d1 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:23.479922 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 999/2012] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:52:23 2022] POST /identity/v3/users => generated 289 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:52:23.481165 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b9e4a-8efe-4fb8-8d2e-e5f5fd92e48d admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:23.491652 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b9e4a-8efe-4fb8-8d2e-e5f5fd92e48d admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:23.496262 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee27076-03ae-4a7b-a764-16eaeb904efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:23.496874 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee27076-03ae-4a7b-a764-16eaeb904efa admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:23.497147 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee27076-03ae-4a7b-a764-16eaeb904efa admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:23.497413 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee27076-03ae-4a7b-a764-16eaeb904efa admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:23.498099 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee27076-03ae-4a7b-a764-16eaeb904efa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:23.499987 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee27076-03ae-4a7b-a764-16eaeb904efa admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:23.511361 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1014/2013] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:52:23.515479 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e162eb2b-36a4-4514-b7df-e13822b564d1 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:23.526301 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e162eb2b-36a4-4514-b7df-e13822b564d1 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:23.531027 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827b021-5bc5-4ba4-ae2b-faf3d09d3db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:23.531631 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827b021-5bc5-4ba4-ae2b-faf3d09d3db1 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:23.531949 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827b021-5bc5-4ba4-ae2b-faf3d09d3db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:23.532280 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827b021-5bc5-4ba4-ae2b-faf3d09d3db1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:23.533107 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5827b021-5bc5-4ba4-ae2b-faf3d09d3db1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:23.534660 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5827b021-5bc5-4ba4-ae2b-faf3d09d3db1 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:23.545817 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1000/2014] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:52:23.549860 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee27076-03ae-4a7b-a764-16eaeb904efa admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:23.560446 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee27076-03ae-4a7b-a764-16eaeb904efa admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:23.565146 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c87427-24b6-4b58-8796-69f884ad81dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:23.565798 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c87427-24b6-4b58-8796-69f884ad81dd admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:23.566077 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c87427-24b6-4b58-8796-69f884ad81dd admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:23.566348 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c87427-24b6-4b58-8796-69f884ad81dd admin admin] PATH_INFO: `/v3/projects/e027ab8bf4024f00bc11ef3b016705c4/users/5931c0f001c74d7f95659c62194e4f98/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:23.582086 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c87427-24b6-4b58-8796-69f884ad81dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e027ab8bf4024f00bc11ef3b016705c4, user_id=5931c0f001c74d7f95659c62194e4f98, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:23.584943 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c87427-24b6-4b58-8796-69f884ad81dd admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:23.601167 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1015/2015] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:52:23 2022] PUT /identity/v3/projects/e027ab8bf4024f00bc11ef3b016705c4/users/5931c0f001c74d7f95659c62194e4f98/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:23.611364 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0bb47-f6dc-4558-b84a-40bdb56f8cf6 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:23.611727 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0bb47-f6dc-4558-b84a-40bdb56f8cf6 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:23.612022 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0bb47-f6dc-4558-b84a-40bdb56f8cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:23.644619 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-4bd0bb47-f6dc-4558-b84a-40bdb56f8cf6 None None] MFA Rules not processed for user `5931c0f001c74d7f95659c62194e4f98`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:23.645979 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-4bd0bb47-f6dc-4558-b84a-40bdb56f8cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:23.665390 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:23.714283 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1001/2016] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:23.719217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c87427-24b6-4b58-8796-69f884ad81dd admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:23.790297 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c87427-24b6-4b58-8796-69f884ad81dd admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:23.795397 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bbde2f-2b75-4753-b105-86db72ae40a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:23.796047 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bbde2f-2b75-4753-b105-86db72ae40a7 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:23.796332 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bbde2f-2b75-4753-b105-86db72ae40a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:23.796612 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bbde2f-2b75-4753-b105-86db72ae40a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:23.797279 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bbde2f-2b75-4753-b105-86db72ae40a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:23.799202 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bbde2f-2b75-4753-b105-86db72ae40a7 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:23.839209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1016/2017] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:52:23 2022] POST /identity/v3/projects => generated 377 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:52:23.840665 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0bb47-f6dc-4558-b84a-40bdb56f8cf6 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:23.852278 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0bb47-f6dc-4558-b84a-40bdb56f8cf6 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:23.915013 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b6993b-9911-4706-9acd-2f0b80fde53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:23.915654 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b6993b-9911-4706-9acd-2f0b80fde53f admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:23.915950 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b6993b-9911-4706-9acd-2f0b80fde53f admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:23.916279 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b6993b-9911-4706-9acd-2f0b80fde53f admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:23.916914 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b6993b-9911-4706-9acd-2f0b80fde53f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:23.918848 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b6993b-9911-4706-9acd-2f0b80fde53f admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:23.946919 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1002/2018] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:52:23 2022] POST /identity/v3/users => generated 295 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:52:23.950923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bbde2f-2b75-4753-b105-86db72ae40a7 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:23.963986 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bbde2f-2b75-4753-b105-86db72ae40a7 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:23.972309 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0dd90-8f71-4e5e-adfd-eb888ab3d0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:23.973325 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0dd90-8f71-4e5e-adfd-eb888ab3d0d3 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:23.973767 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0dd90-8f71-4e5e-adfd-eb888ab3d0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:23.974068 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0dd90-8f71-4e5e-adfd-eb888ab3d0d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:23.975475 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e0dd90-8f71-4e5e-adfd-eb888ab3d0d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:23.977186 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e0dd90-8f71-4e5e-adfd-eb888ab3d0d3 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:23.989186 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1017/2019] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:52:23.995821 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b6993b-9911-4706-9acd-2f0b80fde53f admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:24.014009 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b6993b-9911-4706-9acd-2f0b80fde53f admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:24.018895 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acba302-5dac-4009-8182-af1238bb2562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:24.019534 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acba302-5dac-4009-8182-af1238bb2562 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:24.019851 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acba302-5dac-4009-8182-af1238bb2562 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:24.020127 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acba302-5dac-4009-8182-af1238bb2562 admin admin] PATH_INFO: `/v3/projects/254ea2bcfc7340e3b136d56c19dbf78f/users/3017ce9343dd4089a7bfc40e7d31c731/roles/0e78444e43c744668178971af4087ca0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:24.036667 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acba302-5dac-4009-8182-af1238bb2562 admin admin] RBAC: Authorizing `identity:create_grant(project_id=254ea2bcfc7340e3b136d56c19dbf78f, user_id=3017ce9343dd4089a7bfc40e7d31c731, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:24.040070 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acba302-5dac-4009-8182-af1238bb2562 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:24.054039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1003/2020] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:52:23 2022] PUT /identity/v3/projects/254ea2bcfc7340e3b136d56c19dbf78f/users/3017ce9343dd4089a7bfc40e7d31c731/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:52:24.057953 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0dd90-8f71-4e5e-adfd-eb888ab3d0d3 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:24.068372 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0dd90-8f71-4e5e-adfd-eb888ab3d0d3 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:24.097638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:24.124801 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b9f75-0e69-40bf-8635-ba2b7291923d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:24.125409 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b9f75-0e69-40bf-8635-ba2b7291923d admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:24.125687 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b9f75-0e69-40bf-8635-ba2b7291923d admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:24.125954 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b9f75-0e69-40bf-8635-ba2b7291923d admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:24.126634 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9b9f75-0e69-40bf-8635-ba2b7291923d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:24.128247 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9b9f75-0e69-40bf-8635-ba2b7291923d admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:24.137781 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1018/2021] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:52:24.142119 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acba302-5dac-4009-8182-af1238bb2562 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:24.152927 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acba302-5dac-4009-8182-af1238bb2562 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:24.157605 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b8259-c2aa-4400-b225-77d2e70145be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:24.158245 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b8259-c2aa-4400-b225-77d2e70145be admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:24.158526 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b8259-c2aa-4400-b225-77d2e70145be admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:24.158799 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b8259-c2aa-4400-b225-77d2e70145be admin admin] PATH_INFO: `/v3/projects/254ea2bcfc7340e3b136d56c19dbf78f/users/3017ce9343dd4089a7bfc40e7d31c731/roles/0e78444e43c744668178971af4087ca0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:24.160816 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67b8259-c2aa-4400-b225-77d2e70145be admin admin] RBAC: Authorizing `identity:create_grant(project_id=254ea2bcfc7340e3b136d56c19dbf78f, user_id=3017ce9343dd4089a7bfc40e7d31c731, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:24.164136 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67b8259-c2aa-4400-b225-77d2e70145be admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:24.183778 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1004/2022] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:52:24 2022] PUT /identity/v3/projects/254ea2bcfc7340e3b136d56c19dbf78f/users/3017ce9343dd4089a7bfc40e7d31c731/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:24.185997 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813fc10d-603f-4e90-80d8-b137b251e7a7 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:24.186289 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813fc10d-603f-4e90-80d8-b137b251e7a7 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:24.186583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813fc10d-603f-4e90-80d8-b137b251e7a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:24.216858 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-813fc10d-603f-4e90-80d8-b137b251e7a7 None None] MFA Rules not processed for user `3017ce9343dd4089a7bfc40e7d31c731`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:24.218049 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-813fc10d-603f-4e90-80d8-b137b251e7a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:24.289604 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1019/2023] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:24 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:24.304645 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b8259-c2aa-4400-b225-77d2e70145be admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:24.380725 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b8259-c2aa-4400-b225-77d2e70145be admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:24.384407 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9ea15-a156-4c1b-b035-9920271c7030 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:24.385022 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9ea15-a156-4c1b-b035-9920271c7030 service cinder] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:24.385301 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9ea15-a156-4c1b-b035-9920271c7030 service cinder] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:24.385573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9ea15-a156-4c1b-b035-9920271c7030 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:24.411223 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:24.446049 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a9ea15-a156-4c1b-b035-9920271c7030 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:24.447502 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a9ea15-a156-4c1b-b035-9920271c7030 service cinder] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:24.456699 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1005/2024] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:52:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:52:28.098850 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813fc10d-603f-4e90-80d8-b137b251e7a7 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:28.100791 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-813fc10d-603f-4e90-80d8-b137b251e7a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:28.114405 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813fc10d-603f-4e90-80d8-b137b251e7a7 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:28.118911 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28d77c-7233-4966-819b-5122ad19ab08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:28.119617 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28d77c-7233-4966-819b-5122ad19ab08 service cinder] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:28.119935 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28d77c-7233-4966-819b-5122ad19ab08 service cinder] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:28.120202 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28d77c-7233-4966-819b-5122ad19ab08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:28.192119 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff28d77c-7233-4966-819b-5122ad19ab08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:28.194246 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff28d77c-7233-4966-819b-5122ad19ab08 service cinder] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:28.208241 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1020/2025] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:52:28 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:52:28.810360 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9ea15-a156-4c1b-b035-9920271c7030 service cinder] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:28.819687 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9ea15-a156-4c1b-b035-9920271c7030 service cinder] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:28.823357 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60ed02a-4f2f-4b7b-8151-9acca117e796 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:28.823959 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ed02a-4f2f-4b7b-8151-9acca117e796 service cinder] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:28.824233 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ed02a-4f2f-4b7b-8151-9acca117e796 service cinder] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:28.824502 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ed02a-4f2f-4b7b-8151-9acca117e796 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:28.833534 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60ed02a-4f2f-4b7b-8151-9acca117e796 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:28.834991 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60ed02a-4f2f-4b7b-8151-9acca117e796 service cinder] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:28.844904 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1006/2026] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:52:28 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:52:28.934459 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28d77c-7233-4966-819b-5122ad19ab08 service cinder] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:28.935980 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-ff28d77c-7233-4966-819b-5122ad19ab08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:29.018218 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:29.037055 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28d77c-7233-4966-819b-5122ad19ab08 service cinder] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:29.044097 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43830a4a-5aa0-4970-993c-e8c36fcc270d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:29.044789 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43830a4a-5aa0-4970-993c-e8c36fcc270d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:29.045241 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43830a4a-5aa0-4970-993c-e8c36fcc270d admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:29.045842 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43830a4a-5aa0-4970-993c-e8c36fcc270d admin admin] PATH_INFO: `/v3/users/ccf959c2ccd64ae19f2f4c6134d078af` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:29.048517 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43830a4a-5aa0-4970-993c-e8c36fcc270d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccf959c2ccd64ae19f2f4c6134d078af)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:29.050908 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43830a4a-5aa0-4970-993c-e8c36fcc270d admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:29.109435 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-43830a4a-5aa0-4970-993c-e8c36fcc270d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccf959c2ccd64ae19f2f4c6134d078af'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:29.124842 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-43830a4a-5aa0-4970-993c-e8c36fcc270d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccf959c2ccd64ae19f2f4c6134d078af'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:29.125589 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-43830a4a-5aa0-4970-993c-e8c36fcc270d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccf959c2ccd64ae19f2f4c6134d078af'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:29.134510 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-43830a4a-5aa0-4970-993c-e8c36fcc270d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccf959c2ccd64ae19f2f4c6134d078af'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:29.152411 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-43830a4a-5aa0-4970-993c-e8c36fcc270d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccf959c2ccd64ae19f2f4c6134d078af'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:29.159086 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1021/2027] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:52:28 2022] DELETE /identity/v3/users/ccf959c2ccd64ae19f2f4c6134d078af => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:29.165220 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76929f-24da-4745-904e-2946a1b26f31 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:29.165605 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76929f-24da-4745-904e-2946a1b26f31 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:29.165893 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76929f-24da-4745-904e-2946a1b26f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:29.191969 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-0f76929f-24da-4745-904e-2946a1b26f31 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:29.193124 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-0f76929f-24da-4745-904e-2946a1b26f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:29.255329 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1007/2028] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:29.265464 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43830a4a-5aa0-4970-993c-e8c36fcc270d admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:29.266884 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-43830a4a-5aa0-4970-993c-e8c36fcc270d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:29.327141 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43830a4a-5aa0-4970-993c-e8c36fcc270d admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:29.330839 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792a1c2c-8a84-4294-baca-cd019163098e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:29.331477 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a1c2c-8a84-4294-baca-cd019163098e service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:29.331786 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a1c2c-8a84-4294-baca-cd019163098e service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:29.332081 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a1c2c-8a84-4294-baca-cd019163098e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:29.342326 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792a1c2c-8a84-4294-baca-cd019163098e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:29.343828 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792a1c2c-8a84-4294-baca-cd019163098e service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:29.353696 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1022/2029] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:52:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:52:29.663539 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f76929f-24da-4745-904e-2946a1b26f31 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:29.665433 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-0f76929f-24da-4745-904e-2946a1b26f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:29.680593 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f76929f-24da-4745-904e-2946a1b26f31 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:29.686374 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f7bd1-4c38-4711-904c-10f9bf797027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:29.687138 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f7bd1-4c38-4711-904c-10f9bf797027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:29.687513 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f7bd1-4c38-4711-904c-10f9bf797027 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:29.687909 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f7bd1-4c38-4711-904c-10f9bf797027 admin admin] PATH_INFO: `/v3/projects/36edefec489849368c6b2849d86fbd36` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:29.689365 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1f7bd1-4c38-4711-904c-10f9bf797027 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36edefec489849368c6b2849d86fbd36)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:29.691500 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1f7bd1-4c38-4711-904c-10f9bf797027 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:29.745066 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-6e1f7bd1-4c38-4711-904c-10f9bf797027 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36edefec489849368c6b2849d86fbd36', 'request_id': 'req-6e1f7bd1-4c38-4711-904c-10f9bf797027'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:29.752256 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-6e1f7bd1-4c38-4711-904c-10f9bf797027 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36edefec489849368c6b2849d86fbd36', 'request_id': 'req-6e1f7bd1-4c38-4711-904c-10f9bf797027'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:29.760623 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-6e1f7bd1-4c38-4711-904c-10f9bf797027 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36edefec489849368c6b2849d86fbd36', 'request_id': 'req-6e1f7bd1-4c38-4711-904c-10f9bf797027'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:29.772856 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-6e1f7bd1-4c38-4711-904c-10f9bf797027 admin admin] The token cache is being invalidate because project 36edefec489849368c6b2849d86fbd36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:52:29.773164 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-6e1f7bd1-4c38-4711-904c-10f9bf797027 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:29.773799 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-6e1f7bd1-4c38-4711-904c-10f9bf797027 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:29.775336 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1008/2030] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:52:29 2022] DELETE /identity/v3/projects/36edefec489849368c6b2849d86fbd36 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:29.779443 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792a1c2c-8a84-4294-baca-cd019163098e service neutron] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:29.789060 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792a1c2c-8a84-4294-baca-cd019163098e service neutron] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:29.851446 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441e6a8e-ccdc-4ba0-8086-aeb19791099b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:29.852128 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441e6a8e-ccdc-4ba0-8086-aeb19791099b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:29.852407 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441e6a8e-ccdc-4ba0-8086-aeb19791099b admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:29.852682 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441e6a8e-ccdc-4ba0-8086-aeb19791099b admin admin] PATH_INFO: `/v3/users/854e32c1babc4608b5af4be1c026f6bb` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:29.854210 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441e6a8e-ccdc-4ba0-8086-aeb19791099b admin admin] RBAC: Authorizing `identity:delete_user(user_id=854e32c1babc4608b5af4be1c026f6bb)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:29.856956 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441e6a8e-ccdc-4ba0-8086-aeb19791099b admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:29.874075 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:29.916228 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-441e6a8e-ccdc-4ba0-8086-aeb19791099b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '854e32c1babc4608b5af4be1c026f6bb'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:29.924842 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-441e6a8e-ccdc-4ba0-8086-aeb19791099b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '854e32c1babc4608b5af4be1c026f6bb'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:29.925534 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-441e6a8e-ccdc-4ba0-8086-aeb19791099b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '854e32c1babc4608b5af4be1c026f6bb'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:29.933048 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-441e6a8e-ccdc-4ba0-8086-aeb19791099b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '854e32c1babc4608b5af4be1c026f6bb'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:29.950045 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-441e6a8e-ccdc-4ba0-8086-aeb19791099b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '854e32c1babc4608b5af4be1c026f6bb'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:29.956865 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1023/2031] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:52:29 2022] DELETE /identity/v3/users/854e32c1babc4608b5af4be1c026f6bb => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:52:30.572021 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f7bd1-4c38-4711-904c-10f9bf797027 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:30.576166 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-6e1f7bd1-4c38-4711-904c-10f9bf797027 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:30.590599 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f7bd1-4c38-4711-904c-10f9bf797027 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:30.632032 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:30.669552 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef002218-5182-4ba1-8de3-8c7b28c77fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:30.670467 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef002218-5182-4ba1-8de3-8c7b28c77fa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:30.670870 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef002218-5182-4ba1-8de3-8c7b28c77fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:30.671401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef002218-5182-4ba1-8de3-8c7b28c77fa5 admin admin] PATH_INFO: `/v3/projects/834f850d7aba40cabb76a6d79eb649de` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:30.672663 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef002218-5182-4ba1-8de3-8c7b28c77fa5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=834f850d7aba40cabb76a6d79eb649de)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:30.674608 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef002218-5182-4ba1-8de3-8c7b28c77fa5 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:30.724071 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-ef002218-5182-4ba1-8de3-8c7b28c77fa5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '834f850d7aba40cabb76a6d79eb649de', 'request_id': 'req-ef002218-5182-4ba1-8de3-8c7b28c77fa5'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:30.733827 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-ef002218-5182-4ba1-8de3-8c7b28c77fa5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '834f850d7aba40cabb76a6d79eb649de', 'request_id': 'req-ef002218-5182-4ba1-8de3-8c7b28c77fa5'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:30.744627 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-ef002218-5182-4ba1-8de3-8c7b28c77fa5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '834f850d7aba40cabb76a6d79eb649de', 'request_id': 'req-ef002218-5182-4ba1-8de3-8c7b28c77fa5'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:30.758123 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-ef002218-5182-4ba1-8de3-8c7b28c77fa5 admin admin] The token cache is being invalidate because project 834f850d7aba40cabb76a6d79eb649de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:52:30.758123 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-ef002218-5182-4ba1-8de3-8c7b28c77fa5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:30.758812 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-ef002218-5182-4ba1-8de3-8c7b28c77fa5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:30.763792 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1009/2032] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:52:30 2022] DELETE /identity/v3/projects/834f850d7aba40cabb76a6d79eb649de => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:30.765678 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32f1be-8c40-4acd-8738-3e3e748e39fc None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:30.766166 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32f1be-8c40-4acd-8738-3e3e748e39fc None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:30.766385 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32f1be-8c40-4acd-8738-3e3e748e39fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:30.790040 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-1c32f1be-8c40-4acd-8738-3e3e748e39fc None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:30.791422 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-1c32f1be-8c40-4acd-8738-3e3e748e39fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:30.854499 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1024/2033] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:30.863520 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668ccf3-9515-4caa-aac7-4912b8b536ec None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:30.863878 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668ccf3-9515-4caa-aac7-4912b8b536ec None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:30.864328 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668ccf3-9515-4caa-aac7-4912b8b536ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:30.887103 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-5668ccf3-9515-4caa-aac7-4912b8b536ec None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:30.888473 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-5668ccf3-9515-4caa-aac7-4912b8b536ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:30.897136 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1010/2034] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:30.902968 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c32f1be-8c40-4acd-8738-3e3e748e39fc None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:30.913809 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c32f1be-8c40-4acd-8738-3e3e748e39fc None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:30.918928 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1fd70a-153c-4b00-a3e6-3ac9dd5c74cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:30.919546 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1fd70a-153c-4b00-a3e6-3ac9dd5c74cf admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:30.919868 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1fd70a-153c-4b00-a3e6-3ac9dd5c74cf admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:30.920145 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1fd70a-153c-4b00-a3e6-3ac9dd5c74cf admin admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:30.920903 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1fd70a-153c-4b00-a3e6-3ac9dd5c74cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:52:30.921181 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1fd70a-153c-4b00-a3e6-3ac9dd5c74cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:30.923096 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1fd70a-153c-4b00-a3e6-3ac9dd5c74cf admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:30.937762 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1025/2035] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:52:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:52:30.942308 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5668ccf3-9515-4caa-aac7-4912b8b536ec None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:30.954622 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5668ccf3-9515-4caa-aac7-4912b8b536ec None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:30.961200 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a820e9-546b-478b-9c92-7fd7ee313584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:30.961935 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a820e9-546b-478b-9c92-7fd7ee313584 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:30.962248 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a820e9-546b-478b-9c92-7fd7ee313584 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:30.962529 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a820e9-546b-478b-9c92-7fd7ee313584 admin admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:30.963247 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a820e9-546b-478b-9c92-7fd7ee313584 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:30.965355 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a820e9-546b-478b-9c92-7fd7ee313584 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:30.999252 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1011/2036] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:52:30 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:52:31.000637 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1fd70a-153c-4b00-a3e6-3ac9dd5c74cf admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:31.013181 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1fd70a-153c-4b00-a3e6-3ac9dd5c74cf admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:31.074376 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91733537-08bf-4f32-b915-c43e82fb9603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:31.074995 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91733537-08bf-4f32-b915-c43e82fb9603 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:31.075269 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91733537-08bf-4f32-b915-c43e82fb9603 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:31.075539 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91733537-08bf-4f32-b915-c43e82fb9603 admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:31.076255 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91733537-08bf-4f32-b915-c43e82fb9603 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:31.078175 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91733537-08bf-4f32-b915-c43e82fb9603 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:31.101833 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1026/2037] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:52:30 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:52:31.106276 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a820e9-546b-478b-9c92-7fd7ee313584 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:31.116995 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a820e9-546b-478b-9c92-7fd7ee313584 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:31.121782 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e8279-d5eb-453a-83c8-3ade734ae9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:31.122458 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e8279-d5eb-453a-83c8-3ade734ae9c3 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:31.122666 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e8279-d5eb-453a-83c8-3ade734ae9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:31.123003 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e8279-d5eb-453a-83c8-3ade734ae9c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:31.124018 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e8279-d5eb-453a-83c8-3ade734ae9c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:31.125807 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e8279-d5eb-453a-83c8-3ade734ae9c3 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:31.140760 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1012/2038] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:52:31.144910 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91733537-08bf-4f32-b915-c43e82fb9603 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:31.157267 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91733537-08bf-4f32-b915-c43e82fb9603 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:31.167254 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b95d4-6859-4009-9630-5e3065e18135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:31.168019 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b95d4-6859-4009-9630-5e3065e18135 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:31.168300 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b95d4-6859-4009-9630-5e3065e18135 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:31.168570 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b95d4-6859-4009-9630-5e3065e18135 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:31.169277 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51b95d4-6859-4009-9630-5e3065e18135 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:31.170917 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51b95d4-6859-4009-9630-5e3065e18135 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:31.178061 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:31.185443 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1027/2039] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:31 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:52:31.189439 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e8279-d5eb-453a-83c8-3ade734ae9c3 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:31.200354 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e8279-d5eb-453a-83c8-3ade734ae9c3 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:31.206865 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b6e6bd-22e5-4530-887c-992d6c337e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:31.207528 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b6e6bd-22e5-4530-887c-992d6c337e73 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:31.208047 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b6e6bd-22e5-4530-887c-992d6c337e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:31.208398 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b6e6bd-22e5-4530-887c-992d6c337e73 admin admin] PATH_INFO: `/v3/projects/f4a7b397de474d8d8efd60806c21fd87/users/8532893896e14659a5a6b8d9b929f899/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:31.230025 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b6e6bd-22e5-4530-887c-992d6c337e73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4a7b397de474d8d8efd60806c21fd87, user_id=8532893896e14659a5a6b8d9b929f899, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:31.232965 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b6e6bd-22e5-4530-887c-992d6c337e73 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:31.249031 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1013/2040] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:52:31 2022] PUT /identity/v3/projects/f4a7b397de474d8d8efd60806c21fd87/users/8532893896e14659a5a6b8d9b929f899/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:31.261395 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc011adc-d0d0-46fb-98af-3c8faa7b4856 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:31.261679 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc011adc-d0d0-46fb-98af-3c8faa7b4856 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:31.261979 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc011adc-d0d0-46fb-98af-3c8faa7b4856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:31.291110 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-bc011adc-d0d0-46fb-98af-3c8faa7b4856 None None] MFA Rules not processed for user `8532893896e14659a5a6b8d9b929f899`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:31.292361 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-bc011adc-d0d0-46fb-98af-3c8faa7b4856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:31.380993 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1028/2041] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:31.392372 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b6e6bd-22e5-4530-887c-992d6c337e73 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:31.394783 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-98b6e6bd-22e5-4530-887c-992d6c337e73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:31.489036 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b6e6bd-22e5-4530-887c-992d6c337e73 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:31.493212 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18751a1d-1335-462e-a46d-bccc3077f31d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:31.493970 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18751a1d-1335-462e-a46d-bccc3077f31d service cinder] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:31.494358 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18751a1d-1335-462e-a46d-bccc3077f31d service cinder] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:31.494748 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18751a1d-1335-462e-a46d-bccc3077f31d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:31.505432 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18751a1d-1335-462e-a46d-bccc3077f31d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:31.507194 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18751a1d-1335-462e-a46d-bccc3077f31d service cinder] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:31.518496 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1014/2042] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:52:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:52:32.496918 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72845ff-9ea7-4d11-be09-75889c7ad087 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:32.497518 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72845ff-9ea7-4d11-be09-75889c7ad087 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:32.497518 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72845ff-9ea7-4d11-be09-75889c7ad087 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:32.499906 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1029/2043] 10.222.0.20 () {54 vars in 921 bytes} [Tue Jul 26 12:52:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:52:32.526122 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d515f8d9-6cc4-4d96-9fdd-fce15483cbd5 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:32.526755 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d515f8d9-6cc4-4d96-9fdd-fce15483cbd5 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:32.526847 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d515f8d9-6cc4-4d96-9fdd-fce15483cbd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:32.536809 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:32.552878 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-d515f8d9-6cc4-4d96-9fdd-fce15483cbd5 None None] MFA Rules not processed for user `ec49a03aed774c439459883bd5442d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:32.557140 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-d515f8d9-6cc4-4d96-9fdd-fce15483cbd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:32.645709 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1015/2044] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Jul 26 12:52:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:52:32.656988 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72845ff-9ea7-4d11-be09-75889c7ad087 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:32.658964 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-a72845ff-9ea7-4d11-be09-75889c7ad087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:32.676851 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72845ff-9ea7-4d11-be09-75889c7ad087 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:32.682740 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabad467-aa7a-4c8d-a482-44dd3c1e7d5c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec49a03aed774c439459883bd5442d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:32.683449 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabad467-aa7a-4c8d-a482-44dd3c1e7d5c service placement] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:32.683830 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabad467-aa7a-4c8d-a482-44dd3c1e7d5c service placement] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:32.684169 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabad467-aa7a-4c8d-a482-44dd3c1e7d5c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:32.693052 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabad467-aa7a-4c8d-a482-44dd3c1e7d5c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:32.694703 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabad467-aa7a-4c8d-a482-44dd3c1e7d5c service placement] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:32.704813 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1030/2045] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:52:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:52:33.218028 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d515f8d9-6cc4-4d96-9fdd-fce15483cbd5 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:33.219811 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-d515f8d9-6cc4-4d96-9fdd-fce15483cbd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:33.309097 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d515f8d9-6cc4-4d96-9fdd-fce15483cbd5 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:33.313691 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe2efc8-bc14-4ce9-a6b7-46f2899ff76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:33.314341 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2efc8-bc14-4ce9-a6b7-46f2899ff76c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:33.314461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabad467-aa7a-4c8d-a482-44dd3c1e7d5c service placement] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:33.314778 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2efc8-bc14-4ce9-a6b7-46f2899ff76c admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:33.315178 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2efc8-bc14-4ce9-a6b7-46f2899ff76c admin admin] PATH_INFO: `/v3/users/c6f439a6212643248d050040d4eac29a` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:33.316389 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe2efc8-bc14-4ce9-a6b7-46f2899ff76c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6f439a6212643248d050040d4eac29a)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:33.316509 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-dabad467-aa7a-4c8d-a482-44dd3c1e7d5c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:33.318909 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe2efc8-bc14-4ce9-a6b7-46f2899ff76c admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:33.351004 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabad467-aa7a-4c8d-a482-44dd3c1e7d5c service placement] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:33.357575 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865caaf8-97c1-4599-8c8d-b2b94d40bb88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:33.358584 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865caaf8-97c1-4599-8c8d-b2b94d40bb88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:33.359176 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865caaf8-97c1-4599-8c8d-b2b94d40bb88 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:33.359868 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865caaf8-97c1-4599-8c8d-b2b94d40bb88 admin admin] PATH_INFO: `/v3/users/eea3bddaa92f48aa867e0f97f99a990a` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:33.362020 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865caaf8-97c1-4599-8c8d-b2b94d40bb88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eea3bddaa92f48aa867e0f97f99a990a)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:33.364434 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865caaf8-97c1-4599-8c8d-b2b94d40bb88 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:33.394812 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-7fe2efc8-bc14-4ce9-a6b7-46f2899ff76c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6f439a6212643248d050040d4eac29a'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:33.395868 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-7fe2efc8-bc14-4ce9-a6b7-46f2899ff76c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6f439a6212643248d050040d4eac29a'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:33.397152 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-7fe2efc8-bc14-4ce9-a6b7-46f2899ff76c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6f439a6212643248d050040d4eac29a'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:33.415789 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-7fe2efc8-bc14-4ce9-a6b7-46f2899ff76c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6f439a6212643248d050040d4eac29a'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:33.424919 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-7fe2efc8-bc14-4ce9-a6b7-46f2899ff76c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6f439a6212643248d050040d4eac29a'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:33.451583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-865caaf8-97c1-4599-8c8d-b2b94d40bb88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eea3bddaa92f48aa867e0f97f99a990a'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:33.459105 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1016/2046] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:52:33 2022] DELETE /identity/v3/users/c6f439a6212643248d050040d4eac29a => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:33.463678 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-865caaf8-97c1-4599-8c8d-b2b94d40bb88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eea3bddaa92f48aa867e0f97f99a990a'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:33.464318 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-865caaf8-97c1-4599-8c8d-b2b94d40bb88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eea3bddaa92f48aa867e0f97f99a990a'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:33.474054 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-865caaf8-97c1-4599-8c8d-b2b94d40bb88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eea3bddaa92f48aa867e0f97f99a990a'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:33.491269 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-865caaf8-97c1-4599-8c8d-b2b94d40bb88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eea3bddaa92f48aa867e0f97f99a990a'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:33.502095 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1031/2047] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:52:33 2022] DELETE /identity/v3/users/eea3bddaa92f48aa867e0f97f99a990a => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:33.507244 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182634d-ac2e-4fc1-9369-ce9ecfc16b37 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:33.507535 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182634d-ac2e-4fc1-9369-ce9ecfc16b37 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:33.507844 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182634d-ac2e-4fc1-9369-ce9ecfc16b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:33.533486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-2182634d-ac2e-4fc1-9369-ce9ecfc16b37 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:33.534714 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-2182634d-ac2e-4fc1-9369-ce9ecfc16b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:33.551914 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:33.631060 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1017/2048] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:33.646250 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865caaf8-97c1-4599-8c8d-b2b94d40bb88 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:33.647725 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-865caaf8-97c1-4599-8c8d-b2b94d40bb88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:33.722569 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:33.728539 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865caaf8-97c1-4599-8c8d-b2b94d40bb88 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:33.733981 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5390c6-73ac-4b35-b089-c96d856d9686 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:33.734905 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5390c6-73ac-4b35-b089-c96d856d9686 service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:33.735211 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5390c6-73ac-4b35-b089-c96d856d9686 service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:33.735679 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5390c6-73ac-4b35-b089-c96d856d9686 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:33.749616 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5390c6-73ac-4b35-b089-c96d856d9686 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:33.751114 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5390c6-73ac-4b35-b089-c96d856d9686 service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:33.763687 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1032/2049] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:52:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:52:33.766840 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2182634d-ac2e-4fc1-9369-ce9ecfc16b37 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:33.768309 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-2182634d-ac2e-4fc1-9369-ce9ecfc16b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:33.781371 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2182634d-ac2e-4fc1-9369-ce9ecfc16b37 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:33.795297 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c1dd6-8d5b-40bc-b52f-4e0ed37a881b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:33.796108 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c1dd6-8d5b-40bc-b52f-4e0ed37a881b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:33.796468 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c1dd6-8d5b-40bc-b52f-4e0ed37a881b admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:33.796805 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c1dd6-8d5b-40bc-b52f-4e0ed37a881b admin admin] PATH_INFO: `/v3/projects/ff00ee711e2b42b9beaf866b10cc8f6f` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:33.798128 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802c1dd6-8d5b-40bc-b52f-4e0ed37a881b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff00ee711e2b42b9beaf866b10cc8f6f)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:33.800195 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802c1dd6-8d5b-40bc-b52f-4e0ed37a881b admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:33.852622 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-802c1dd6-8d5b-40bc-b52f-4e0ed37a881b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff00ee711e2b42b9beaf866b10cc8f6f', 'request_id': 'req-802c1dd6-8d5b-40bc-b52f-4e0ed37a881b'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:33.858764 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-802c1dd6-8d5b-40bc-b52f-4e0ed37a881b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff00ee711e2b42b9beaf866b10cc8f6f', 'request_id': 'req-802c1dd6-8d5b-40bc-b52f-4e0ed37a881b'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:33.867145 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-802c1dd6-8d5b-40bc-b52f-4e0ed37a881b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff00ee711e2b42b9beaf866b10cc8f6f', 'request_id': 'req-802c1dd6-8d5b-40bc-b52f-4e0ed37a881b'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:33.878638 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-802c1dd6-8d5b-40bc-b52f-4e0ed37a881b admin admin] The token cache is being invalidate because project ff00ee711e2b42b9beaf866b10cc8f6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:52:33.878936 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-802c1dd6-8d5b-40bc-b52f-4e0ed37a881b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:33.879595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-802c1dd6-8d5b-40bc-b52f-4e0ed37a881b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:33.881853 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1018/2050] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:52:33 2022] DELETE /identity/v3/projects/ff00ee711e2b42b9beaf866b10cc8f6f => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:33.885982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5390c6-73ac-4b35-b089-c96d856d9686 service neutron] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:33.896187 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5390c6-73ac-4b35-b089-c96d856d9686 service neutron] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:33.961899 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c16c12-2e92-42d7-9736-534bcda60699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:33.962694 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c16c12-2e92-42d7-9736-534bcda60699 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:33.963044 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c16c12-2e92-42d7-9736-534bcda60699 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:33.963464 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c16c12-2e92-42d7-9736-534bcda60699 admin admin] PATH_INFO: `/v3/users/59deeab927c44656bd4ff7adaacf762a` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:33.964752 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c16c12-2e92-42d7-9736-534bcda60699 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59deeab927c44656bd4ff7adaacf762a)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:33.966776 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c16c12-2e92-42d7-9736-534bcda60699 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:34.021196 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-71c16c12-2e92-42d7-9736-534bcda60699 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59deeab927c44656bd4ff7adaacf762a'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:34.030279 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-71c16c12-2e92-42d7-9736-534bcda60699 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59deeab927c44656bd4ff7adaacf762a'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:34.030975 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-71c16c12-2e92-42d7-9736-534bcda60699 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59deeab927c44656bd4ff7adaacf762a'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:34.038150 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-71c16c12-2e92-42d7-9736-534bcda60699 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59deeab927c44656bd4ff7adaacf762a'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:34.052838 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-71c16c12-2e92-42d7-9736-534bcda60699 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59deeab927c44656bd4ff7adaacf762a'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:34.059663 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1033/2051] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:52:33 2022] DELETE /identity/v3/users/59deeab927c44656bd4ff7adaacf762a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:52:34.066457 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c1dd6-8d5b-40bc-b52f-4e0ed37a881b admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:34.068477 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-802c1dd6-8d5b-40bc-b52f-4e0ed37a881b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:34.081834 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c1dd6-8d5b-40bc-b52f-4e0ed37a881b admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:34.177737 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c009c43-12c7-4e8f-ad93-d3567828ce47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:34.178448 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c009c43-12c7-4e8f-ad93-d3567828ce47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:34.178736 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c009c43-12c7-4e8f-ad93-d3567828ce47 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:34.179016 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c009c43-12c7-4e8f-ad93-d3567828ce47 admin admin] PATH_INFO: `/v3/projects/ed3db84b8f2f4091875947bb77911b6f` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:34.180443 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c009c43-12c7-4e8f-ad93-d3567828ce47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed3db84b8f2f4091875947bb77911b6f)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:34.183106 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c009c43-12c7-4e8f-ad93-d3567828ce47 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:34.231689 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:34.235254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-3c009c43-12c7-4e8f-ad93-d3567828ce47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed3db84b8f2f4091875947bb77911b6f', 'request_id': 'req-3c009c43-12c7-4e8f-ad93-d3567828ce47'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:34.242789 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-3c009c43-12c7-4e8f-ad93-d3567828ce47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed3db84b8f2f4091875947bb77911b6f', 'request_id': 'req-3c009c43-12c7-4e8f-ad93-d3567828ce47'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:34.251493 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-3c009c43-12c7-4e8f-ad93-d3567828ce47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed3db84b8f2f4091875947bb77911b6f', 'request_id': 'req-3c009c43-12c7-4e8f-ad93-d3567828ce47'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:34.264232 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-3c009c43-12c7-4e8f-ad93-d3567828ce47 admin admin] The token cache is being invalidate because project ed3db84b8f2f4091875947bb77911b6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:52:34.264547 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-3c009c43-12c7-4e8f-ad93-d3567828ce47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:34.265205 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-3c009c43-12c7-4e8f-ad93-d3567828ce47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:34.266794 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1019/2052] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:52:34 2022] DELETE /identity/v3/projects/ed3db84b8f2f4091875947bb77911b6f => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:34.270919 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c16c12-2e92-42d7-9736-534bcda60699 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:34.272615 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-71c16c12-2e92-42d7-9736-534bcda60699 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:34.287744 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c16c12-2e92-42d7-9736-534bcda60699 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:34.358465 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c009c43-12c7-4e8f-ad93-d3567828ce47 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:34.359958 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-3c009c43-12c7-4e8f-ad93-d3567828ce47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:34.361680 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54bbd42-6273-4a8f-bde3-8ef924b6b286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:34.362373 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54bbd42-6273-4a8f-bde3-8ef924b6b286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:34.362597 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54bbd42-6273-4a8f-bde3-8ef924b6b286 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:34.362860 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54bbd42-6273-4a8f-bde3-8ef924b6b286 admin admin] PATH_INFO: `/v3/users/5931c0f001c74d7f95659c62194e4f98` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:34.364110 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54bbd42-6273-4a8f-bde3-8ef924b6b286 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5931c0f001c74d7f95659c62194e4f98)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:34.366697 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54bbd42-6273-4a8f-bde3-8ef924b6b286 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:34.377050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c009c43-12c7-4e8f-ad93-d3567828ce47 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:34.383442 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcde082-5ade-4ea3-b0f8-7fe6df27c21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:34.384151 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcde082-5ade-4ea3-b0f8-7fe6df27c21e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:34.384440 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcde082-5ade-4ea3-b0f8-7fe6df27c21e admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:34.384721 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcde082-5ade-4ea3-b0f8-7fe6df27c21e admin admin] PATH_INFO: `/v3/projects/761b59a0b7184c2c814c83c78986ec0d` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:34.385951 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcde082-5ade-4ea3-b0f8-7fe6df27c21e admin admin] RBAC: Authorizing `identity:delete_project(project_id=761b59a0b7184c2c814c83c78986ec0d)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:34.388039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcde082-5ade-4ea3-b0f8-7fe6df27c21e admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:34.388449 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:34.428937 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-c54bbd42-6273-4a8f-bde3-8ef924b6b286 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5931c0f001c74d7f95659c62194e4f98'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:34.436446 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-c54bbd42-6273-4a8f-bde3-8ef924b6b286 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5931c0f001c74d7f95659c62194e4f98'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:34.437144 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-c54bbd42-6273-4a8f-bde3-8ef924b6b286 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5931c0f001c74d7f95659c62194e4f98'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:34.442071 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-2bcde082-5ade-4ea3-b0f8-7fe6df27c21e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '761b59a0b7184c2c814c83c78986ec0d', 'request_id': 'req-2bcde082-5ade-4ea3-b0f8-7fe6df27c21e'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:34.446163 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-c54bbd42-6273-4a8f-bde3-8ef924b6b286 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5931c0f001c74d7f95659c62194e4f98'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:34.452766 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-2bcde082-5ade-4ea3-b0f8-7fe6df27c21e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '761b59a0b7184c2c814c83c78986ec0d', 'request_id': 'req-2bcde082-5ade-4ea3-b0f8-7fe6df27c21e'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:34.461571 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-c54bbd42-6273-4a8f-bde3-8ef924b6b286 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5931c0f001c74d7f95659c62194e4f98'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:34.462711 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-2bcde082-5ade-4ea3-b0f8-7fe6df27c21e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '761b59a0b7184c2c814c83c78986ec0d', 'request_id': 'req-2bcde082-5ade-4ea3-b0f8-7fe6df27c21e'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:34.468155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1034/2053] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:52:34 2022] DELETE /identity/v3/users/5931c0f001c74d7f95659c62194e4f98 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:34.473973 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-2bcde082-5ade-4ea3-b0f8-7fe6df27c21e admin admin] The token cache is being invalidate because project 761b59a0b7184c2c814c83c78986ec0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:52:34.474825 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-2bcde082-5ade-4ea3-b0f8-7fe6df27c21e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:34.476107 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-2bcde082-5ade-4ea3-b0f8-7fe6df27c21e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:34.477458 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1020/2054] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:52:34 2022] DELETE /identity/v3/projects/761b59a0b7184c2c814c83c78986ec0d => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:52:34.482299 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8609f5a7-b82b-4b3c-b133-804e635db1ce None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:34.482587 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8609f5a7-b82b-4b3c-b133-804e635db1ce None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:34.482884 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8609f5a7-b82b-4b3c-b133-804e635db1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:34.502329 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-8609f5a7-b82b-4b3c-b133-804e635db1ce None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:34.503542 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-8609f5a7-b82b-4b3c-b133-804e635db1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:34.564714 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1035/2055] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:52:34.573537 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b5d26-08e0-45f4-8c16-c07287e1df72 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:34.574004 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b5d26-08e0-45f4-8c16-c07287e1df72 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:34.574421 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b5d26-08e0-45f4-8c16-c07287e1df72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:34.595303 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-451b5d26-08e0-45f4-8c16-c07287e1df72 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:34.596587 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-451b5d26-08e0-45f4-8c16-c07287e1df72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:34.605550 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1021/2056] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:34.609559 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8609f5a7-b82b-4b3c-b133-804e635db1ce None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:34.619378 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8609f5a7-b82b-4b3c-b133-804e635db1ce None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:34.624349 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e80ebd7-7cb8-4c5f-be08-aaa31c2c7749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:34.624973 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e80ebd7-7cb8-4c5f-be08-aaa31c2c7749 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:34.625253 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e80ebd7-7cb8-4c5f-be08-aaa31c2c7749 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:34.625529 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e80ebd7-7cb8-4c5f-be08-aaa31c2c7749 admin admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:34.626232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e80ebd7-7cb8-4c5f-be08-aaa31c2c7749 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:52:34.626535 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e80ebd7-7cb8-4c5f-be08-aaa31c2c7749 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:34.628185 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e80ebd7-7cb8-4c5f-be08-aaa31c2c7749 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:34.641273 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1036/2057] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:52:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:52:34.645430 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b5d26-08e0-45f4-8c16-c07287e1df72 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:34.655940 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b5d26-08e0-45f4-8c16-c07287e1df72 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:34.664207 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ed5b1-78b0-4e36-8af8-c817faebe062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:34.664834 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ed5b1-78b0-4e36-8af8-c817faebe062 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:34.665111 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ed5b1-78b0-4e36-8af8-c817faebe062 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:34.665387 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ed5b1-78b0-4e36-8af8-c817faebe062 admin admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:34.666516 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819ed5b1-78b0-4e36-8af8-c817faebe062 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:34.668505 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819ed5b1-78b0-4e36-8af8-c817faebe062 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:34.702563 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1022/2058] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:52:34 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:52:34.703814 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e80ebd7-7cb8-4c5f-be08-aaa31c2c7749 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:34.714908 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e80ebd7-7cb8-4c5f-be08-aaa31c2c7749 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:34.772028 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ed5b1-78b0-4e36-8af8-c817faebe062 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:34.773754 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-819ed5b1-78b0-4e36-8af8-c817faebe062 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:34.804344 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c609bc97-1f3a-4524-bb5c-55c9017df545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:34.804984 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c609bc97-1f3a-4524-bb5c-55c9017df545 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:34.805284 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c609bc97-1f3a-4524-bb5c-55c9017df545 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:34.805554 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c609bc97-1f3a-4524-bb5c-55c9017df545 admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:34.806237 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c609bc97-1f3a-4524-bb5c-55c9017df545 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:34.808478 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c609bc97-1f3a-4524-bb5c-55c9017df545 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:34.834832 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1037/2059] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:52:34 2022] POST /identity/v3/users => generated 288 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:52:34.838481 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c609bc97-1f3a-4524-bb5c-55c9017df545 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:34.848756 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c609bc97-1f3a-4524-bb5c-55c9017df545 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:34.854716 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2912c0a-fc39-48e8-a775-7193748d54d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:34.855329 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2912c0a-fc39-48e8-a775-7193748d54d6 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:34.855615 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2912c0a-fc39-48e8-a775-7193748d54d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:34.855921 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2912c0a-fc39-48e8-a775-7193748d54d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:34.856622 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2912c0a-fc39-48e8-a775-7193748d54d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:34.858243 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2912c0a-fc39-48e8-a775-7193748d54d6 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:34.859966 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:34.874946 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ed5b1-78b0-4e36-8af8-c817faebe062 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:34.875764 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1038/2060] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 12:52:34.879251 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2912c0a-fc39-48e8-a775-7193748d54d6 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:34.879831 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b7b8c8-b3e7-4107-85d3-9b14b5d9b893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:34.880546 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b7b8c8-b3e7-4107-85d3-9b14b5d9b893 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:34.880831 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b7b8c8-b3e7-4107-85d3-9b14b5d9b893 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:34.881106 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b7b8c8-b3e7-4107-85d3-9b14b5d9b893 admin admin] PATH_INFO: `/v3/projects/e027ab8bf4024f00bc11ef3b016705c4` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:34.882263 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b7b8c8-b3e7-4107-85d3-9b14b5d9b893 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e027ab8bf4024f00bc11ef3b016705c4)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:34.884234 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b7b8c8-b3e7-4107-85d3-9b14b5d9b893 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:34.889624 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2912c0a-fc39-48e8-a775-7193748d54d6 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:34.895255 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8038e69d-c10d-4449-a2d1-0c55a2a21377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:34.896266 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038e69d-c10d-4449-a2d1-0c55a2a21377 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:34.896388 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038e69d-c10d-4449-a2d1-0c55a2a21377 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:34.896661 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038e69d-c10d-4449-a2d1-0c55a2a21377 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:34.897366 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8038e69d-c10d-4449-a2d1-0c55a2a21377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:34.898987 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8038e69d-c10d-4449-a2d1-0c55a2a21377 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:34.912557 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1039/2061] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 12:52:34.917872 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8038e69d-c10d-4449-a2d1-0c55a2a21377 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:34.924613 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:34.929819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8038e69d-c10d-4449-a2d1-0c55a2a21377 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:34.941129 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-71b7b8c8-b3e7-4107-85d3-9b14b5d9b893 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e027ab8bf4024f00bc11ef3b016705c4', 'request_id': 'req-71b7b8c8-b3e7-4107-85d3-9b14b5d9b893'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:34.947627 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-71b7b8c8-b3e7-4107-85d3-9b14b5d9b893 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e027ab8bf4024f00bc11ef3b016705c4', 'request_id': 'req-71b7b8c8-b3e7-4107-85d3-9b14b5d9b893'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:34.956441 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-71b7b8c8-b3e7-4107-85d3-9b14b5d9b893 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e027ab8bf4024f00bc11ef3b016705c4', 'request_id': 'req-71b7b8c8-b3e7-4107-85d3-9b14b5d9b893'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:34.969003 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-71b7b8c8-b3e7-4107-85d3-9b14b5d9b893 admin admin] The token cache is being invalidate because project e027ab8bf4024f00bc11ef3b016705c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:52:34.969290 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-71b7b8c8-b3e7-4107-85d3-9b14b5d9b893 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:34.969979 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-71b7b8c8-b3e7-4107-85d3-9b14b5d9b893 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:34.972018 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1023/2062] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:52:34 2022] DELETE /identity/v3/projects/e027ab8bf4024f00bc11ef3b016705c4 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:52:34.976766 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b7b8c8-b3e7-4107-85d3-9b14b5d9b893 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:34.978060 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-71b7b8c8-b3e7-4107-85d3-9b14b5d9b893 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:34.989753 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adde37cf-8b20-4a28-a2b5-da5b52a08ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:34.990915 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde37cf-8b20-4a28-a2b5-da5b52a08ed9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:34.991199 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde37cf-8b20-4a28-a2b5-da5b52a08ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:34.991477 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde37cf-8b20-4a28-a2b5-da5b52a08ed9 admin admin] PATH_INFO: `/v3/projects/20bc5fcf58914d7bbfa1d1be1327f4d3/users/8b1779409bff4e1fa6610bedd641b8d9/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:35.008271 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adde37cf-8b20-4a28-a2b5-da5b52a08ed9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20bc5fcf58914d7bbfa1d1be1327f4d3, user_id=8b1779409bff4e1fa6610bedd641b8d9, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:35.011206 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adde37cf-8b20-4a28-a2b5-da5b52a08ed9 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:35.024464 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1040/2063] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:52:34 2022] PUT /identity/v3/projects/20bc5fcf58914d7bbfa1d1be1327f4d3/users/8b1779409bff4e1fa6610bedd641b8d9/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:52:35.036068 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11946f-8e6b-4dd5-838b-7cf5f90e1380 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:35.036348 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11946f-8e6b-4dd5-838b-7cf5f90e1380 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:35.037174 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11946f-8e6b-4dd5-838b-7cf5f90e1380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:35.046614 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b7b8c8-b3e7-4107-85d3-9b14b5d9b893 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:35.065848 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-9f11946f-8e6b-4dd5-838b-7cf5f90e1380 None None] MFA Rules not processed for user `8b1779409bff4e1fa6610bedd641b8d9`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:35.067076 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-9f11946f-8e6b-4dd5-838b-7cf5f90e1380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:35.106275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6400a1-5355-42cf-b8b7-a59d7edd2f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:35.106941 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6400a1-5355-42cf-b8b7-a59d7edd2f5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:35.107226 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6400a1-5355-42cf-b8b7-a59d7edd2f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:35.107519 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6400a1-5355-42cf-b8b7-a59d7edd2f5e admin admin] PATH_INFO: `/v3/users/3017ce9343dd4089a7bfc40e7d31c731` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:35.110302 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6400a1-5355-42cf-b8b7-a59d7edd2f5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3017ce9343dd4089a7bfc40e7d31c731)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:35.114206 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6400a1-5355-42cf-b8b7-a59d7edd2f5e admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:35.150940 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1041/2064] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:52:35.162624 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f11946f-8e6b-4dd5-838b-7cf5f90e1380 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:35.164065 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-9f11946f-8e6b-4dd5-838b-7cf5f90e1380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:35.174483 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-ef6400a1-5355-42cf-b8b7-a59d7edd2f5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3017ce9343dd4089a7bfc40e7d31c731'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:35.175178 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-ef6400a1-5355-42cf-b8b7-a59d7edd2f5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3017ce9343dd4089a7bfc40e7d31c731'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:35.182228 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-ef6400a1-5355-42cf-b8b7-a59d7edd2f5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3017ce9343dd4089a7bfc40e7d31c731'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:35.198560 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-ef6400a1-5355-42cf-b8b7-a59d7edd2f5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3017ce9343dd4089a7bfc40e7d31c731'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:35.206189 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:35.214996 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-ef6400a1-5355-42cf-b8b7-a59d7edd2f5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3017ce9343dd4089a7bfc40e7d31c731'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:35.229741 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1024/2065] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:52:34 2022] DELETE /identity/v3/users/3017ce9343dd4089a7bfc40e7d31c731 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:52:35.245888 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:35.262457 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f11946f-8e6b-4dd5-838b-7cf5f90e1380 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:35.321580 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e8c8b-1be4-4213-a241-334e31b466f6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2b5853838dc40db89c46038c0606107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:35.322211 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e8c8b-1be4-4213-a241-334e31b466f6 service glance] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:35.322506 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e8c8b-1be4-4213-a241-334e31b466f6 service glance] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:35.322789 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e8c8b-1be4-4213-a241-334e31b466f6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:35.324629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-aa3e8c8b-1be4-4213-a241-334e31b466f6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:35.394087 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3e8c8b-1be4-4213-a241-334e31b466f6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:35.395588 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3e8c8b-1be4-4213-a241-334e31b466f6 service glance] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:35.405304 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1042/2066] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:52:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 12:52:35.558911 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6400a1-5355-42cf-b8b7-a59d7edd2f5e admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:35.560972 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-ef6400a1-5355-42cf-b8b7-a59d7edd2f5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:35.642394 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6400a1-5355-42cf-b8b7-a59d7edd2f5e admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:35.647110 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58737922-3906-472d-9495-1ebcac9a4d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:35.647780 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58737922-3906-472d-9495-1ebcac9a4d24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:35.648062 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58737922-3906-472d-9495-1ebcac9a4d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:35.648333 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58737922-3906-472d-9495-1ebcac9a4d24 admin admin] PATH_INFO: `/v3/projects/254ea2bcfc7340e3b136d56c19dbf78f` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:35.649575 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58737922-3906-472d-9495-1ebcac9a4d24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=254ea2bcfc7340e3b136d56c19dbf78f)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:35.651512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58737922-3906-472d-9495-1ebcac9a4d24 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:35.700968 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-58737922-3906-472d-9495-1ebcac9a4d24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '254ea2bcfc7340e3b136d56c19dbf78f', 'request_id': 'req-58737922-3906-472d-9495-1ebcac9a4d24'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:35.707144 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-58737922-3906-472d-9495-1ebcac9a4d24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '254ea2bcfc7340e3b136d56c19dbf78f', 'request_id': 'req-58737922-3906-472d-9495-1ebcac9a4d24'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:35.718156 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-58737922-3906-472d-9495-1ebcac9a4d24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '254ea2bcfc7340e3b136d56c19dbf78f', 'request_id': 'req-58737922-3906-472d-9495-1ebcac9a4d24'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:35.730446 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-58737922-3906-472d-9495-1ebcac9a4d24 admin admin] The token cache is being invalidate because project 254ea2bcfc7340e3b136d56c19dbf78f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:52:35.730731 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-58737922-3906-472d-9495-1ebcac9a4d24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:35.731383 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-58737922-3906-472d-9495-1ebcac9a4d24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:35.732933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1025/2067] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:52:35 2022] DELETE /identity/v3/projects/254ea2bcfc7340e3b136d56c19dbf78f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:35.738425 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ec32d-e5a6-4239-acd0-86e4197b4d4d None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:35.738934 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ec32d-e5a6-4239-acd0-86e4197b4d4d None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:35.739024 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ec32d-e5a6-4239-acd0-86e4197b4d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:35.761951 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-f77ec32d-e5a6-4239-acd0-86e4197b4d4d None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:35.763322 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-f77ec32d-e5a6-4239-acd0-86e4197b4d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:35.832591 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1043/2068] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:35.841676 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc6fcb-e678-4bca-b3c7-f2e7c935912c None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:35.842074 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc6fcb-e678-4bca-b3c7-f2e7c935912c None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:35.842292 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc6fcb-e678-4bca-b3c7-f2e7c935912c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:35.874194 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-9bcc6fcb-e678-4bca-b3c7-f2e7c935912c None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:35.875579 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-9bcc6fcb-e678-4bca-b3c7-f2e7c935912c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:35.895352 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1026/2069] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:35.899678 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77ec32d-e5a6-4239-acd0-86e4197b4d4d None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:35.910064 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77ec32d-e5a6-4239-acd0-86e4197b4d4d None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:35.915886 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb30d3f8-a26b-408b-a1d4-3ed1e9aaa337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:35.916741 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30d3f8-a26b-408b-a1d4-3ed1e9aaa337 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:35.917058 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30d3f8-a26b-408b-a1d4-3ed1e9aaa337 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:35.917421 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30d3f8-a26b-408b-a1d4-3ed1e9aaa337 admin admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:35.918234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb30d3f8-a26b-408b-a1d4-3ed1e9aaa337 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:52:35.918646 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb30d3f8-a26b-408b-a1d4-3ed1e9aaa337 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:35.920910 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb30d3f8-a26b-408b-a1d4-3ed1e9aaa337 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:35.935826 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1044/2070] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:52:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:52:35.940394 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc6fcb-e678-4bca-b3c7-f2e7c935912c None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:35.951869 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc6fcb-e678-4bca-b3c7-f2e7c935912c None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:35.955695 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ec31b-999a-46b3-bed1-fc2fc521907f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:35.956311 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ec31b-999a-46b3-bed1-fc2fc521907f admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:35.956727 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ec31b-999a-46b3-bed1-fc2fc521907f admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:35.957003 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ec31b-999a-46b3-bed1-fc2fc521907f admin admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:35.957679 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ec31b-999a-46b3-bed1-fc2fc521907f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:35.960390 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ec31b-999a-46b3-bed1-fc2fc521907f admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:35.990889 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1027/2071] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:52:35 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:52:35.995209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb30d3f8-a26b-408b-a1d4-3ed1e9aaa337 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:36.006128 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb30d3f8-a26b-408b-a1d4-3ed1e9aaa337 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:36.040487 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:36.064767 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b1fecf-849a-46a6-bad8-0734b9c4e0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:36.065376 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1fecf-849a-46a6-bad8-0734b9c4e0cc admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:36.065657 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1fecf-849a-46a6-bad8-0734b9c4e0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:36.065930 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1fecf-849a-46a6-bad8-0734b9c4e0cc admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:36.066593 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b1fecf-849a-46a6-bad8-0734b9c4e0cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:36.068592 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b1fecf-849a-46a6-bad8-0734b9c4e0cc admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:36.089759 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1045/2072] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:52:35 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:52:36.093975 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ec31b-999a-46b3-bed1-fc2fc521907f admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:36.104330 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ec31b-999a-46b3-bed1-fc2fc521907f admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:36.108984 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c6656-6f38-4c8b-a52f-0027b1ced1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:36.109586 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c6656-6f38-4c8b-a52f-0027b1ced1ca admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:36.109862 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c6656-6f38-4c8b-a52f-0027b1ced1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:36.110152 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c6656-6f38-4c8b-a52f-0027b1ced1ca admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:36.110875 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580c6656-6f38-4c8b-a52f-0027b1ced1ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:36.112482 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580c6656-6f38-4c8b-a52f-0027b1ced1ca admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:36.122774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1028/2073] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:52:36.127492 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b1fecf-849a-46a6-bad8-0734b9c4e0cc admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:36.138250 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b1fecf-849a-46a6-bad8-0734b9c4e0cc admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:36.143036 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b2a6a7-e438-4750-9ad6-3a2175aee644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:36.143660 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b2a6a7-e438-4750-9ad6-3a2175aee644 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:36.143949 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b2a6a7-e438-4750-9ad6-3a2175aee644 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:36.144227 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b2a6a7-e438-4750-9ad6-3a2175aee644 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:36.144920 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b2a6a7-e438-4750-9ad6-3a2175aee644 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:36.146521 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b2a6a7-e438-4750-9ad6-3a2175aee644 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:36.157382 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1046/2074] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:52:36.161687 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c6656-6f38-4c8b-a52f-0027b1ced1ca admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:36.171876 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c6656-6f38-4c8b-a52f-0027b1ced1ca admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:36.176955 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59058de-b875-43af-9341-4844c27da995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:36.178152 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59058de-b875-43af-9341-4844c27da995 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:36.178436 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59058de-b875-43af-9341-4844c27da995 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:36.178742 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59058de-b875-43af-9341-4844c27da995 admin admin] PATH_INFO: `/v3/projects/2ea32124f33d45c1a667dd6ece8bfb99/users/46360a6ed1384bf8bfc6be98659509f6/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:36.194568 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59058de-b875-43af-9341-4844c27da995 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ea32124f33d45c1a667dd6ece8bfb99, user_id=46360a6ed1384bf8bfc6be98659509f6, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:36.197281 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59058de-b875-43af-9341-4844c27da995 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:36.217271 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1029/2075] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:52:36 2022] PUT /identity/v3/projects/2ea32124f33d45c1a667dd6ece8bfb99/users/46360a6ed1384bf8bfc6be98659509f6/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:52:36.227460 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70117cf3-0eba-4bcd-ae80-73e05c29226b None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:36.227780 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70117cf3-0eba-4bcd-ae80-73e05c29226b None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:36.228059 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70117cf3-0eba-4bcd-ae80-73e05c29226b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:36.258543 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-70117cf3-0eba-4bcd-ae80-73e05c29226b None None] MFA Rules not processed for user `46360a6ed1384bf8bfc6be98659509f6`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:36.259784 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-70117cf3-0eba-4bcd-ae80-73e05c29226b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:36.325188 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1047/2076] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:36.330041 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59058de-b875-43af-9341-4844c27da995 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:36.343195 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:36.398002 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59058de-b875-43af-9341-4844c27da995 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:36.402713 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d151e-022a-405e-aaea-65afabb99ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:36.403322 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d151e-022a-405e-aaea-65afabb99ad2 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:36.403604 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d151e-022a-405e-aaea-65afabb99ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:36.403908 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d151e-022a-405e-aaea-65afabb99ad2 admin admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:36.404591 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24d151e-022a-405e-aaea-65afabb99ad2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:36.406583 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24d151e-022a-405e-aaea-65afabb99ad2 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:36.437719 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1030/2077] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:52:36 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:52:36.442079 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70117cf3-0eba-4bcd-ae80-73e05c29226b None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:36.453075 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70117cf3-0eba-4bcd-ae80-73e05c29226b None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:36.516741 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687572b5-355d-4d58-8577-283fa2a5b2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:36.517427 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687572b5-355d-4d58-8577-283fa2a5b2a6 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:36.517691 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687572b5-355d-4d58-8577-283fa2a5b2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:36.517963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687572b5-355d-4d58-8577-283fa2a5b2a6 admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:36.518636 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687572b5-355d-4d58-8577-283fa2a5b2a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:36.520593 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687572b5-355d-4d58-8577-283fa2a5b2a6 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:36.546801 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1048/2078] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:52:36 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:52:36.548502 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d151e-022a-405e-aaea-65afabb99ad2 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:36.567104 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d151e-022a-405e-aaea-65afabb99ad2 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:36.572833 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b9987f-31e8-4a6d-a95a-f9d2715abf90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:36.573557 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b9987f-31e8-4a6d-a95a-f9d2715abf90 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:36.573819 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b9987f-31e8-4a6d-a95a-f9d2715abf90 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:36.574660 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b9987f-31e8-4a6d-a95a-f9d2715abf90 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:36.574881 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b9987f-31e8-4a6d-a95a-f9d2715abf90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:36.576597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b9987f-31e8-4a6d-a95a-f9d2715abf90 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:36.588431 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1031/2079] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:36 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:52:36.592839 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687572b5-355d-4d58-8577-283fa2a5b2a6 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:36.607984 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687572b5-355d-4d58-8577-283fa2a5b2a6 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:36.612893 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d60d9d-1cc8-4c18-937c-438df71bd4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:36.613589 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d60d9d-1cc8-4c18-937c-438df71bd4f9 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:36.613956 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d60d9d-1cc8-4c18-937c-438df71bd4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:36.614257 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d60d9d-1cc8-4c18-937c-438df71bd4f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:36.615598 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d60d9d-1cc8-4c18-937c-438df71bd4f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:36.623874 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d60d9d-1cc8-4c18-937c-438df71bd4f9 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:36.631837 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1049/2080] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:36 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 12:52:36.636501 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b9987f-31e8-4a6d-a95a-f9d2715abf90 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:36.649256 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b9987f-31e8-4a6d-a95a-f9d2715abf90 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:36.654708 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653de93a-b6d7-48af-af14-d93f91d9b9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:36.654998 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653de93a-b6d7-48af-af14-d93f91d9b9dc admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:36.655214 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653de93a-b6d7-48af-af14-d93f91d9b9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:36.655667 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653de93a-b6d7-48af-af14-d93f91d9b9dc admin admin] PATH_INFO: `/v3/projects/ffee4fe12ff44aafafd6e85c092d8b71/users/eb8196f17d7949c5a5da0a36b03dc33f/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:36.673255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653de93a-b6d7-48af-af14-d93f91d9b9dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffee4fe12ff44aafafd6e85c092d8b71, user_id=eb8196f17d7949c5a5da0a36b03dc33f, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:36.676155 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653de93a-b6d7-48af-af14-d93f91d9b9dc admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:36.690754 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1032/2081] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:52:36 2022] PUT /identity/v3/projects/ffee4fe12ff44aafafd6e85c092d8b71/users/eb8196f17d7949c5a5da0a36b03dc33f/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:36.706159 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca870e-21c1-4e36-a2aa-2c862d890778 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:36.706451 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca870e-21c1-4e36-a2aa-2c862d890778 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:36.706730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca870e-21c1-4e36-a2aa-2c862d890778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:36.741261 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-1bca870e-21c1-4e36-a2aa-2c862d890778 None None] MFA Rules not processed for user `eb8196f17d7949c5a5da0a36b03dc33f`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:36.742515 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-1bca870e-21c1-4e36-a2aa-2c862d890778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:36.753799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:36.838623 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1050/2082] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:36.846211 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653de93a-b6d7-48af-af14-d93f91d9b9dc admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:36.922368 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653de93a-b6d7-48af-af14-d93f91d9b9dc admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:36.931057 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e9621c-7840-4a75-b12a-ba4cf8e9a253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:36.931718 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e9621c-7840-4a75-b12a-ba4cf8e9a253 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:36.931998 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e9621c-7840-4a75-b12a-ba4cf8e9a253 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:36.932272 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e9621c-7840-4a75-b12a-ba4cf8e9a253 admin admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:36.932981 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e9621c-7840-4a75-b12a-ba4cf8e9a253 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:36.935064 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e9621c-7840-4a75-b12a-ba4cf8e9a253 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:36.980193 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1033/2083] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:52:36 2022] POST /identity/v3/projects => generated 383 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:52:36.980923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bca870e-21c1-4e36-a2aa-2c862d890778 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:36.991779 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bca870e-21c1-4e36-a2aa-2c862d890778 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:37.064285 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aaa3a9-2788-4053-960e-3d6eace6ba55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:37.064916 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aaa3a9-2788-4053-960e-3d6eace6ba55 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:37.065198 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aaa3a9-2788-4053-960e-3d6eace6ba55 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:37.065473 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aaa3a9-2788-4053-960e-3d6eace6ba55 admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:37.066157 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aaa3a9-2788-4053-960e-3d6eace6ba55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:37.068721 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aaa3a9-2788-4053-960e-3d6eace6ba55 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:37.095181 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1051/2084] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:52:36 2022] POST /identity/v3/users => generated 298 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:52:37.099422 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e9621c-7840-4a75-b12a-ba4cf8e9a253 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:37.109895 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e9621c-7840-4a75-b12a-ba4cf8e9a253 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:37.115249 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a02b01e-9b2c-4f62-ab09-6e271a17e2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:37.116226 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02b01e-9b2c-4f62-ab09-6e271a17e2d9 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:37.116651 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02b01e-9b2c-4f62-ab09-6e271a17e2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:37.117069 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02b01e-9b2c-4f62-ab09-6e271a17e2d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:37.118124 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a02b01e-9b2c-4f62-ab09-6e271a17e2d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:37.119803 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a02b01e-9b2c-4f62-ab09-6e271a17e2d9 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:37.133506 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1034/2085] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 12:52:37.138656 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aaa3a9-2788-4053-960e-3d6eace6ba55 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:37.153606 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aaa3a9-2788-4053-960e-3d6eace6ba55 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:37.159010 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9262e12f-8cf0-4da2-bd0e-50dced9e1727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:37.159517 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9262e12f-8cf0-4da2-bd0e-50dced9e1727 admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:37.159779 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9262e12f-8cf0-4da2-bd0e-50dced9e1727 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:37.160059 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9262e12f-8cf0-4da2-bd0e-50dced9e1727 admin admin] PATH_INFO: `/v3/projects/8a265396be6f402abd771346e2b47e1f/users/dec6296f540645ae9eb873ac6b05a93d/roles/0e78444e43c744668178971af4087ca0` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:37.182533 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9262e12f-8cf0-4da2-bd0e-50dced9e1727 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a265396be6f402abd771346e2b47e1f, user_id=dec6296f540645ae9eb873ac6b05a93d, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:37.185261 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9262e12f-8cf0-4da2-bd0e-50dced9e1727 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:37.199440 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1052/2086] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:52:37 2022] PUT /identity/v3/projects/8a265396be6f402abd771346e2b47e1f/users/dec6296f540645ae9eb873ac6b05a93d/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:37.203234 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a02b01e-9b2c-4f62-ab09-6e271a17e2d9 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:37.213447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a02b01e-9b2c-4f62-ab09-6e271a17e2d9 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:37.250151 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:37.288756 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca64dcab-0a86-4714-8384-ee4234c9e802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:37.288756 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca64dcab-0a86-4714-8384-ee4234c9e802 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:37.288756 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca64dcab-0a86-4714-8384-ee4234c9e802 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:37.288756 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca64dcab-0a86-4714-8384-ee4234c9e802 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:37.289386 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca64dcab-0a86-4714-8384-ee4234c9e802 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:37.291209 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca64dcab-0a86-4714-8384-ee4234c9e802 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:37.303888 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1035/2087] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:52:37.307171 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9262e12f-8cf0-4da2-bd0e-50dced9e1727 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:37.318696 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9262e12f-8cf0-4da2-bd0e-50dced9e1727 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:37.337795 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bc9318-c832-4118-8918-623a6470d582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:37.338514 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc9318-c832-4118-8918-623a6470d582 admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:37.338797 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc9318-c832-4118-8918-623a6470d582 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:37.339107 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc9318-c832-4118-8918-623a6470d582 admin admin] PATH_INFO: `/v3/projects/8a265396be6f402abd771346e2b47e1f/users/dec6296f540645ae9eb873ac6b05a93d/roles/0e78444e43c744668178971af4087ca0` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:37.341289 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bc9318-c832-4118-8918-623a6470d582 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a265396be6f402abd771346e2b47e1f, user_id=dec6296f540645ae9eb873ac6b05a93d, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:37.344672 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bc9318-c832-4118-8918-623a6470d582 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:37.366677 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1053/2088] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:52:37 2022] PUT /identity/v3/projects/8a265396be6f402abd771346e2b47e1f/users/dec6296f540645ae9eb873ac6b05a93d/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:37.377348 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cf021b-33f4-4d0b-b595-1c8113df8ada None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:37.377767 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cf021b-33f4-4d0b-b595-1c8113df8ada None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:37.378272 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cf021b-33f4-4d0b-b595-1c8113df8ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:37.426073 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-c8cf021b-33f4-4d0b-b595-1c8113df8ada None None] MFA Rules not processed for user `dec6296f540645ae9eb873ac6b05a93d`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:37.428031 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-c8cf021b-33f4-4d0b-b595-1c8113df8ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:37.550731 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1036/2089] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:37 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:37.562738 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bc9318-c832-4118-8918-623a6470d582 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:37.564591 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-53bc9318-c832-4118-8918-623a6470d582 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:37.650134 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bc9318-c832-4118-8918-623a6470d582 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:37.661913 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c78459-b52e-4696-9c44-7c6c9a8e1e11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:37.662830 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c78459-b52e-4696-9c44-7c6c9a8e1e11 service cinder] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:37.663294 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c78459-b52e-4696-9c44-7c6c9a8e1e11 service cinder] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:37.664003 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c78459-b52e-4696-9c44-7c6c9a8e1e11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:37.692356 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:37.753158 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c78459-b52e-4696-9c44-7c6c9a8e1e11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:37.754797 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c78459-b52e-4696-9c44-7c6c9a8e1e11 service cinder] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:37.765496 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1054/2090] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:52:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:52:39.533054 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cf021b-33f4-4d0b-b595-1c8113df8ada None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:39.561344 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cf021b-33f4-4d0b-b595-1c8113df8ada None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:39.566823 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b558e47a-f8e1-4249-9b6f-3394133e9b95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:39.567463 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b558e47a-f8e1-4249-9b6f-3394133e9b95 service cinder] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:39.567814 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b558e47a-f8e1-4249-9b6f-3394133e9b95 service cinder] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:39.568091 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b558e47a-f8e1-4249-9b6f-3394133e9b95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:39.646902 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b558e47a-f8e1-4249-9b6f-3394133e9b95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:39.648406 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b558e47a-f8e1-4249-9b6f-3394133e9b95 service cinder] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:39.658374 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1037/2091] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:52:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:52:40.355896 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c78459-b52e-4696-9c44-7c6c9a8e1e11 service cinder] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:40.357504 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-55c78459-b52e-4696-9c44-7c6c9a8e1e11 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:40.374337 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c78459-b52e-4696-9c44-7c6c9a8e1e11 service cinder] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:40.378808 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad53783-a3c7-4d2b-9e5a-f48b31f627cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:40.379436 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad53783-a3c7-4d2b-9e5a-f48b31f627cc service cinder] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:40.379939 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad53783-a3c7-4d2b-9e5a-f48b31f627cc service cinder] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:40.380879 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad53783-a3c7-4d2b-9e5a-f48b31f627cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:40.431905 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad53783-a3c7-4d2b-9e5a-f48b31f627cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:40.433369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad53783-a3c7-4d2b-9e5a-f48b31f627cc service cinder] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:40.447905 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b558e47a-f8e1-4249-9b6f-3394133e9b95 service cinder] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:40.450138 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-b558e47a-f8e1-4249-9b6f-3394133e9b95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:40.450646 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1055/2092] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:52:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:52:40.579445 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:40.587317 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b558e47a-f8e1-4249-9b6f-3394133e9b95 service cinder] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:40.592383 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0316d19b-b1a4-4c99-9868-9951c8476d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:40.593225 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0316d19b-b1a4-4c99-9868-9951c8476d0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:40.593707 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0316d19b-b1a4-4c99-9868-9951c8476d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:40.594194 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0316d19b-b1a4-4c99-9868-9951c8476d0a admin admin] PATH_INFO: `/v3/users/8532893896e14659a5a6b8d9b929f899` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:40.600395 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0316d19b-b1a4-4c99-9868-9951c8476d0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8532893896e14659a5a6b8d9b929f899)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:40.602676 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0316d19b-b1a4-4c99-9868-9951c8476d0a admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:40.677566 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-0316d19b-b1a4-4c99-9868-9951c8476d0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8532893896e14659a5a6b8d9b929f899'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:40.678257 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-0316d19b-b1a4-4c99-9868-9951c8476d0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8532893896e14659a5a6b8d9b929f899'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:40.679127 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-0316d19b-b1a4-4c99-9868-9951c8476d0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8532893896e14659a5a6b8d9b929f899'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:40.700403 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-0316d19b-b1a4-4c99-9868-9951c8476d0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8532893896e14659a5a6b8d9b929f899'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:40.714782 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-0316d19b-b1a4-4c99-9868-9951c8476d0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8532893896e14659a5a6b8d9b929f899'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:40.739219 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1038/2093] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:52:40 2022] DELETE /identity/v3/users/8532893896e14659a5a6b8d9b929f899 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:40.744236 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192236c9-ddd9-401c-b7f1-68b0b6201546 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:40.744525 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192236c9-ddd9-401c-b7f1-68b0b6201546 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:40.744804 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192236c9-ddd9-401c-b7f1-68b0b6201546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:40.766783 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-192236c9-ddd9-401c-b7f1-68b0b6201546 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:40.768258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-192236c9-ddd9-401c-b7f1-68b0b6201546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:40.873569 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1056/2094] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:40.889301 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0316d19b-b1a4-4c99-9868-9951c8476d0a admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:40.892486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-0316d19b-b1a4-4c99-9868-9951c8476d0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:40.975507 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0316d19b-b1a4-4c99-9868-9951c8476d0a admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:40.985567 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff9492f-496f-4692-8d7b-c962e5ec16e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:40.988499 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9492f-496f-4692-8d7b-c962e5ec16e3 service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:40.988790 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9492f-496f-4692-8d7b-c962e5ec16e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:40.990451 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9492f-496f-4692-8d7b-c962e5ec16e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:41.014133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff9492f-496f-4692-8d7b-c962e5ec16e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:41.015791 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff9492f-496f-4692-8d7b-c962e5ec16e3 service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:41.027463 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1039/2095] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:52:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:52:41.072064 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192236c9-ddd9-401c-b7f1-68b0b6201546 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:41.075948 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-192236c9-ddd9-401c-b7f1-68b0b6201546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:41.092520 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192236c9-ddd9-401c-b7f1-68b0b6201546 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:41.097797 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b642335-bfaf-4613-b332-035de79ee850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:41.098633 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b642335-bfaf-4613-b332-035de79ee850 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:41.099554 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b642335-bfaf-4613-b332-035de79ee850 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:41.100050 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b642335-bfaf-4613-b332-035de79ee850 admin admin] PATH_INFO: `/v3/users/7c4ed2f2f6994ae2bb00d5a502715b4f` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:41.101715 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b642335-bfaf-4613-b332-035de79ee850 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c4ed2f2f6994ae2bb00d5a502715b4f)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:41.106369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b642335-bfaf-4613-b332-035de79ee850 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:41.166943 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-1b642335-bfaf-4613-b332-035de79ee850 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c4ed2f2f6994ae2bb00d5a502715b4f'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:41.176112 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-1b642335-bfaf-4613-b332-035de79ee850 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c4ed2f2f6994ae2bb00d5a502715b4f'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:41.188806 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-1b642335-bfaf-4613-b332-035de79ee850 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c4ed2f2f6994ae2bb00d5a502715b4f'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:41.205167 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-1b642335-bfaf-4613-b332-035de79ee850 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c4ed2f2f6994ae2bb00d5a502715b4f'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:41.219284 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-1b642335-bfaf-4613-b332-035de79ee850 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c4ed2f2f6994ae2bb00d5a502715b4f'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:41.230623 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1057/2096] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:52:41 2022] DELETE /identity/v3/users/7c4ed2f2f6994ae2bb00d5a502715b4f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:41.445896 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff9492f-496f-4692-8d7b-c962e5ec16e3 service neutron] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:41.465006 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff9492f-496f-4692-8d7b-c962e5ec16e3 service neutron] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:41.543015 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe938cde-f2f1-4c07-a4eb-3d7ac0613ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:41.543709 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe938cde-f2f1-4c07-a4eb-3d7ac0613ea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:41.544198 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe938cde-f2f1-4c07-a4eb-3d7ac0613ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:41.544472 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe938cde-f2f1-4c07-a4eb-3d7ac0613ea3 admin admin] PATH_INFO: `/v3/projects/0f35a4b7696f4cdfbb6e6b316352c022` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:41.545707 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe938cde-f2f1-4c07-a4eb-3d7ac0613ea3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f35a4b7696f4cdfbb6e6b316352c022)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:41.547683 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe938cde-f2f1-4c07-a4eb-3d7ac0613ea3 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:41.578924 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:41.601215 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b642335-bfaf-4613-b332-035de79ee850 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:41.603946 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-1b642335-bfaf-4613-b332-035de79ee850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:41.637628 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b642335-bfaf-4613-b332-035de79ee850 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:41.642330 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-fe938cde-f2f1-4c07-a4eb-3d7ac0613ea3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f35a4b7696f4cdfbb6e6b316352c022', 'request_id': 'req-fe938cde-f2f1-4c07-a4eb-3d7ac0613ea3'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:41.650472 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-fe938cde-f2f1-4c07-a4eb-3d7ac0613ea3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f35a4b7696f4cdfbb6e6b316352c022', 'request_id': 'req-fe938cde-f2f1-4c07-a4eb-3d7ac0613ea3'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:41.660049 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-fe938cde-f2f1-4c07-a4eb-3d7ac0613ea3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f35a4b7696f4cdfbb6e6b316352c022', 'request_id': 'req-fe938cde-f2f1-4c07-a4eb-3d7ac0613ea3'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:41.660849 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:41.678748 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-fe938cde-f2f1-4c07-a4eb-3d7ac0613ea3 admin admin] The token cache is being invalidate because project 0f35a4b7696f4cdfbb6e6b316352c022 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:52:41.680219 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-fe938cde-f2f1-4c07-a4eb-3d7ac0613ea3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:41.681790 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-fe938cde-f2f1-4c07-a4eb-3d7ac0613ea3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:41.698219 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1040/2097] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:52:41 2022] DELETE /identity/v3/projects/0f35a4b7696f4cdfbb6e6b316352c022 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:41.701264 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855549ff-5238-475c-b81f-c7767fce858a None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:41.701776 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855549ff-5238-475c-b81f-c7767fce858a None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:41.702154 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855549ff-5238-475c-b81f-c7767fce858a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:41.727101 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86336535-df67-40dc-87d2-048a27b26330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:41.728206 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86336535-df67-40dc-87d2-048a27b26330 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:41.728588 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86336535-df67-40dc-87d2-048a27b26330 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:41.728967 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86336535-df67-40dc-87d2-048a27b26330 admin admin] PATH_INFO: `/v3/projects/f4a7b397de474d8d8efd60806c21fd87` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:41.729815 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-855549ff-5238-475c-b81f-c7767fce858a None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:41.730531 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86336535-df67-40dc-87d2-048a27b26330 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4a7b397de474d8d8efd60806c21fd87)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:41.731068 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-855549ff-5238-475c-b81f-c7767fce858a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:41.732850 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86336535-df67-40dc-87d2-048a27b26330 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:41.739882 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1041/2098] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:41.750947 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facf439d-1612-495d-8b2a-9a995be4d9c9 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:41.750947 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facf439d-1612-495d-8b2a-9a995be4d9c9 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:41.750947 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facf439d-1612-495d-8b2a-9a995be4d9c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:41.773590 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-facf439d-1612-495d-8b2a-9a995be4d9c9 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:41.775211 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-facf439d-1612-495d-8b2a-9a995be4d9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:41.788301 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1042/2099] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:52:41.790369 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facf439d-1612-495d-8b2a-9a995be4d9c9 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:41.798853 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-86336535-df67-40dc-87d2-048a27b26330 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4a7b397de474d8d8efd60806c21fd87', 'request_id': 'req-86336535-df67-40dc-87d2-048a27b26330'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:41.801014 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facf439d-1612-495d-8b2a-9a995be4d9c9 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:41.806281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-86336535-df67-40dc-87d2-048a27b26330 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4a7b397de474d8d8efd60806c21fd87', 'request_id': 'req-86336535-df67-40dc-87d2-048a27b26330'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:41.816251 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-86336535-df67-40dc-87d2-048a27b26330 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4a7b397de474d8d8efd60806c21fd87', 'request_id': 'req-86336535-df67-40dc-87d2-048a27b26330'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:41.834721 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-86336535-df67-40dc-87d2-048a27b26330 admin admin] The token cache is being invalidate because project f4a7b397de474d8d8efd60806c21fd87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:52:41.835028 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-86336535-df67-40dc-87d2-048a27b26330 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:41.835914 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-86336535-df67-40dc-87d2-048a27b26330 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:41.838138 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1058/2100] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:52:41 2022] DELETE /identity/v3/projects/f4a7b397de474d8d8efd60806c21fd87 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:41.843405 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2279712f-f0a3-4306-b5b0-54b55d335aef None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:41.843773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2279712f-f0a3-4306-b5b0-54b55d335aef None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:41.844876 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2279712f-f0a3-4306-b5b0-54b55d335aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:41.869042 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-2279712f-f0a3-4306-b5b0-54b55d335aef None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:41.870414 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-2279712f-f0a3-4306-b5b0-54b55d335aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:41.875558 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d396a-b13c-4869-8635-95130083837b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:41.876965 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d396a-b13c-4869-8635-95130083837b admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:41.877621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d396a-b13c-4869-8635-95130083837b admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:41.878163 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d396a-b13c-4869-8635-95130083837b admin admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:41.878953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d396a-b13c-4869-8635-95130083837b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:52:41.879323 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d396a-b13c-4869-8635-95130083837b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:41.881790 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d396a-b13c-4869-8635-95130083837b admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:41.883593 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1059/2101] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:41.892592 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c51a006-834f-4412-b587-8acb08a1f8d8 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:41.893567 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c51a006-834f-4412-b587-8acb08a1f8d8 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:41.893938 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c51a006-834f-4412-b587-8acb08a1f8d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:41.897329 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1043/2102] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:52:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:52:41.902258 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d396a-b13c-4869-8635-95130083837b admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:41.904222 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-cb7d396a-b13c-4869-8635-95130083837b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:41.917320 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d396a-b13c-4869-8635-95130083837b admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:41.922298 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-1c51a006-834f-4412-b587-8acb08a1f8d8 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:41.922848 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b3e5b-3518-4a11-9e00-57f5a38c34f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:41.923467 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b3e5b-3518-4a11-9e00-57f5a38c34f3 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:41.923816 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b3e5b-3518-4a11-9e00-57f5a38c34f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:41.924039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b3e5b-3518-4a11-9e00-57f5a38c34f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:41.924623 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-1c51a006-834f-4412-b587-8acb08a1f8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:41.924744 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85b3e5b-3518-4a11-9e00-57f5a38c34f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:41.926917 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85b3e5b-3518-4a11-9e00-57f5a38c34f3 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:41.936712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1060/2103] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:41.945468 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c51a006-834f-4412-b587-8acb08a1f8d8 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:41.958848 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c51a006-834f-4412-b587-8acb08a1f8d8 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:41.961291 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1044/2104] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:52:41 2022] POST /identity/v3/projects => generated 363 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:52:41.966642 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b3e5b-3518-4a11-9e00-57f5a38c34f3 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:42.050905 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c2d49b-bc2e-4e39-8215-a976035c7f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:42.052818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2d49b-bc2e-4e39-8215-a976035c7f59 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:42.053172 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2d49b-bc2e-4e39-8215-a976035c7f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:42.053522 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2d49b-bc2e-4e39-8215-a976035c7f59 admin admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:42.054290 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c2d49b-bc2e-4e39-8215-a976035c7f59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:52:42.054603 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c2d49b-bc2e-4e39-8215-a976035c7f59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:42.055623 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b3e5b-3518-4a11-9e00-57f5a38c34f3 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:42.056258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c2d49b-bc2e-4e39-8215-a976035c7f59 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:42.060512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f212ebb-70fb-4c24-a6f1-cec525ebdba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:42.061178 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f212ebb-70fb-4c24-a6f1-cec525ebdba2 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:42.061458 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f212ebb-70fb-4c24-a6f1-cec525ebdba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:42.061732 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f212ebb-70fb-4c24-a6f1-cec525ebdba2 admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:42.062453 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f212ebb-70fb-4c24-a6f1-cec525ebdba2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:42.064436 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f212ebb-70fb-4c24-a6f1-cec525ebdba2 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:42.069008 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1061/2105] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:52:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:52:42.073530 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c2d49b-bc2e-4e39-8215-a976035c7f59 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:42.077291 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:42.085876 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c2d49b-bc2e-4e39-8215-a976035c7f59 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:42.089535 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1045/2106] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:52:41 2022] POST /identity/v3/users => generated 282 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:52:42.093065 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8429df8-f77a-400f-8e15-a88fda909896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:42.093269 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f212ebb-70fb-4c24-a6f1-cec525ebdba2 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:42.093691 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8429df8-f77a-400f-8e15-a88fda909896 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:42.094004 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8429df8-f77a-400f-8e15-a88fda909896 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:42.094444 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8429df8-f77a-400f-8e15-a88fda909896 admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:42.095123 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8429df8-f77a-400f-8e15-a88fda909896 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:42.097225 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8429df8-f77a-400f-8e15-a88fda909896 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:42.102791 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f212ebb-70fb-4c24-a6f1-cec525ebdba2 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:42.107386 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b389fac-c40e-4464-85e5-f99e39514fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:42.108050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b389fac-c40e-4464-85e5-f99e39514fcf admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:42.108328 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b389fac-c40e-4464-85e5-f99e39514fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:42.108598 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b389fac-c40e-4464-85e5-f99e39514fcf admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:42.109285 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b389fac-c40e-4464-85e5-f99e39514fcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:42.110857 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b389fac-c40e-4464-85e5-f99e39514fcf admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:42.123922 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1046/2107] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 12:52:42.128116 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b389fac-c40e-4464-85e5-f99e39514fcf admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:42.132210 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1062/2108] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:52:42 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:52:42.133123 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8429df8-f77a-400f-8e15-a88fda909896 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:42.139708 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b389fac-c40e-4464-85e5-f99e39514fcf admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:42.141878 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:42.215904 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8429df8-f77a-400f-8e15-a88fda909896 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:42.216143 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3683bc5b-4be0-461e-bf56-36a3f71391e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:42.216775 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683bc5b-4be0-461e-bf56-36a3f71391e1 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:42.217106 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683bc5b-4be0-461e-bf56-36a3f71391e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:42.217574 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683bc5b-4be0-461e-bf56-36a3f71391e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:42.218303 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3683bc5b-4be0-461e-bf56-36a3f71391e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:42.220026 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3683bc5b-4be0-461e-bf56-36a3f71391e1 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:42.225314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b979f5a8-1b31-4363-99aa-cd134be1bf9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:42.225944 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979f5a8-1b31-4363-99aa-cd134be1bf9f admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:42.226229 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979f5a8-1b31-4363-99aa-cd134be1bf9f admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:42.226539 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979f5a8-1b31-4363-99aa-cd134be1bf9f admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:42.227323 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b979f5a8-1b31-4363-99aa-cd134be1bf9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:42.229459 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b979f5a8-1b31-4363-99aa-cd134be1bf9f admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:42.230784 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1047/2109] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:52:42.234981 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3683bc5b-4be0-461e-bf56-36a3f71391e1 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:42.245783 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3683bc5b-4be0-461e-bf56-36a3f71391e1 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:42.251315 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d03dc-8bc2-41a9-83a9-b53eaea78779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:42.252048 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d03dc-8bc2-41a9-83a9-b53eaea78779 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:42.252305 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d03dc-8bc2-41a9-83a9-b53eaea78779 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:42.252577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d03dc-8bc2-41a9-83a9-b53eaea78779 admin admin] PATH_INFO: `/v3/projects/b609a950c1014a87a80b0cdf93338340/users/2e72882a22074cf980dc72c427cb39f9/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:42.253428 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1063/2110] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:52:42 2022] POST /identity/v3/users => generated 289 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:52:42.260734 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b979f5a8-1b31-4363-99aa-cd134be1bf9f admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:42.271258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b979f5a8-1b31-4363-99aa-cd134be1bf9f admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:42.271420 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d03dc-8bc2-41a9-83a9-b53eaea78779 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b609a950c1014a87a80b0cdf93338340, user_id=2e72882a22074cf980dc72c427cb39f9, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:42.274097 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d03dc-8bc2-41a9-83a9-b53eaea78779 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:42.276252 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b03f96-7842-49ad-b7b3-aa1aedf61bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:42.276856 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b03f96-7842-49ad-b7b3-aa1aedf61bd9 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:42.277132 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b03f96-7842-49ad-b7b3-aa1aedf61bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:42.277400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b03f96-7842-49ad-b7b3-aa1aedf61bd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:42.278094 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b03f96-7842-49ad-b7b3-aa1aedf61bd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:42.279920 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b03f96-7842-49ad-b7b3-aa1aedf61bd9 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:42.287595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1048/2111] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:52:42 2022] PUT /identity/v3/projects/b609a950c1014a87a80b0cdf93338340/users/2e72882a22074cf980dc72c427cb39f9/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:52:42.296399 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1064/2112] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:52:42.296516 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d03dc-8bc2-41a9-83a9-b53eaea78779 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:42.297799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0066e9-9529-49f9-bf8f-d81afade6123 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:42.298081 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0066e9-9529-49f9-bf8f-d81afade6123 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:42.298581 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0066e9-9529-49f9-bf8f-d81afade6123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:42.327680 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-1a0066e9-9529-49f9-bf8f-d81afade6123 None None] MFA Rules not processed for user `2e72882a22074cf980dc72c427cb39f9`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:42.328939 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-1a0066e9-9529-49f9-bf8f-d81afade6123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:42.360503 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d03dc-8bc2-41a9-83a9-b53eaea78779 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:42.366585 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf5c847-c410-4e79-8113-d94898146c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:42.367263 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf5c847-c410-4e79-8113-d94898146c44 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:42.367575 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf5c847-c410-4e79-8113-d94898146c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:42.367924 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf5c847-c410-4e79-8113-d94898146c44 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:42.368643 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf5c847-c410-4e79-8113-d94898146c44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:42.370330 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf5c847-c410-4e79-8113-d94898146c44 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:42.383981 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1049/2113] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:52:42.385322 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf5c847-c410-4e79-8113-d94898146c44 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:42.400384 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf5c847-c410-4e79-8113-d94898146c44 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:42.402159 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1065/2114] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:42 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:52:42.405229 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210f16aa-c073-4236-bf73-bbec589883a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:42.405891 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f16aa-c073-4236-bf73-bbec589883a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:42.406169 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f16aa-c073-4236-bf73-bbec589883a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:42.406440 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f16aa-c073-4236-bf73-bbec589883a9 admin admin] PATH_INFO: `/v3/projects/32882d8e5b2445dabf15c6d081077b0f/users/e482fc46d40f45f58b295012beb17ba6/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:42.406900 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0066e9-9529-49f9-bf8f-d81afade6123 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:42.419546 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0066e9-9529-49f9-bf8f-d81afade6123 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:42.424122 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210f16aa-c073-4236-bf73-bbec589883a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32882d8e5b2445dabf15c6d081077b0f, user_id=e482fc46d40f45f58b295012beb17ba6, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:42.424296 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3c6697-d034-4eef-ba81-5f9cebceb17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:42.424918 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c6697-d034-4eef-ba81-5f9cebceb17c admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:42.425201 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c6697-d034-4eef-ba81-5f9cebceb17c admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:42.425480 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c6697-d034-4eef-ba81-5f9cebceb17c admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:42.426151 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3c6697-d034-4eef-ba81-5f9cebceb17c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:42.427904 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210f16aa-c073-4236-bf73-bbec589883a9 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:42.429093 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3c6697-d034-4eef-ba81-5f9cebceb17c admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:42.443982 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1050/2115] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:52:42 2022] PUT /identity/v3/projects/32882d8e5b2445dabf15c6d081077b0f/users/e482fc46d40f45f58b295012beb17ba6/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:52:42.455682 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d46344d-3f96-4878-aeea-e5a9ceb2038f None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:42.456249 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d46344d-3f96-4878-aeea-e5a9ceb2038f None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:42.456599 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d46344d-3f96-4878-aeea-e5a9ceb2038f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:42.474894 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1066/2116] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:52:42 2022] POST /identity/v3/projects => generated 359 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:52:42.478666 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3c6697-d034-4eef-ba81-5f9cebceb17c admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:42.498560 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-5d46344d-3f96-4878-aeea-e5a9ceb2038f None None] MFA Rules not processed for user `e482fc46d40f45f58b295012beb17ba6`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:42.500097 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-5d46344d-3f96-4878-aeea-e5a9ceb2038f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:42.519354 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:42.539674 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:42.553173 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3c6697-d034-4eef-ba81-5f9cebceb17c admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:42.565108 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d08bf-fefd-49c3-bb7e-fb17dd07c3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:42.565840 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d08bf-fefd-49c3-bb7e-fb17dd07c3ad admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:42.566128 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d08bf-fefd-49c3-bb7e-fb17dd07c3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:42.566388 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d08bf-fefd-49c3-bb7e-fb17dd07c3ad admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:42.567064 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d08bf-fefd-49c3-bb7e-fb17dd07c3ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:42.569199 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d08bf-fefd-49c3-bb7e-fb17dd07c3ad admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:42.579210 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1051/2117] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:52:42.591879 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d46344d-3f96-4878-aeea-e5a9ceb2038f None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:42.595213 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1067/2118] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:52:42 2022] POST /identity/v3/users => generated 286 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:52:42.596138 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-5d46344d-3f96-4878-aeea-e5a9ceb2038f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:42.596462 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d08bf-fefd-49c3-bb7e-fb17dd07c3ad admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:42.607070 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d08bf-fefd-49c3-bb7e-fb17dd07c3ad admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:42.612120 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566baea9-35fb-4820-8098-9c9e55313fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:42.612732 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566baea9-35fb-4820-8098-9c9e55313fd8 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:42.613007 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566baea9-35fb-4820-8098-9c9e55313fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:42.613279 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566baea9-35fb-4820-8098-9c9e55313fd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:42.613974 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566baea9-35fb-4820-8098-9c9e55313fd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:42.616236 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566baea9-35fb-4820-8098-9c9e55313fd8 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:42.626786 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1068/2119] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:52:42.630982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566baea9-35fb-4820-8098-9c9e55313fd8 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:42.640666 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566baea9-35fb-4820-8098-9c9e55313fd8 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:42.645146 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9804260-65cd-4baa-a82f-840e2a39c4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:42.645857 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9804260-65cd-4baa-a82f-840e2a39c4a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:42.646136 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9804260-65cd-4baa-a82f-840e2a39c4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:42.646401 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9804260-65cd-4baa-a82f-840e2a39c4a4 admin admin] PATH_INFO: `/v3/projects/3177e18c06c04a5b89098b72d8ad19b1/users/0483a08230d3487e88b1b36e7bd19727/roles/0e78444e43c744668178971af4087ca0` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:42.661961 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9804260-65cd-4baa-a82f-840e2a39c4a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3177e18c06c04a5b89098b72d8ad19b1, user_id=0483a08230d3487e88b1b36e7bd19727, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:42.662141 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d46344d-3f96-4878-aeea-e5a9ceb2038f None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:42.664700 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9804260-65cd-4baa-a82f-840e2a39c4a4 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:42.666185 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5312d5db-f49b-4c54-a671-07d883b9975c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:42.666961 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312d5db-f49b-4c54-a671-07d883b9975c service cinder] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:42.667270 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312d5db-f49b-4c54-a671-07d883b9975c service cinder] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:42.667546 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312d5db-f49b-4c54-a671-07d883b9975c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:42.677699 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5312d5db-f49b-4c54-a671-07d883b9975c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:42.679189 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5312d5db-f49b-4c54-a671-07d883b9975c service cinder] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:42.679679 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1069/2120] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:52:42 2022] PUT /identity/v3/projects/3177e18c06c04a5b89098b72d8ad19b1/users/0483a08230d3487e88b1b36e7bd19727/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:52:42.683529 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9804260-65cd-4baa-a82f-840e2a39c4a4 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:42.689316 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1052/2121] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:52:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 12:52:42.768936 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9804260-65cd-4baa-a82f-840e2a39c4a4 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:42.777726 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bb45c6-1a38-4932-8c43-e42e5ca721fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:42.778638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb45c6-1a38-4932-8c43-e42e5ca721fd admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:42.779023 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb45c6-1a38-4932-8c43-e42e5ca721fd admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:42.779401 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb45c6-1a38-4932-8c43-e42e5ca721fd admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:42.780255 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bb45c6-1a38-4932-8c43-e42e5ca721fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:42.783687 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bb45c6-1a38-4932-8c43-e42e5ca721fd admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:42.793214 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1070/2122] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:52:42.797289 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5312d5db-f49b-4c54-a671-07d883b9975c service cinder] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:42.810009 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5312d5db-f49b-4c54-a671-07d883b9975c service cinder] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:42.814393 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ed5ee1-f4cf-4659-a7e3-6eb154e51cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:42.815057 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed5ee1-f4cf-4659-a7e3-6eb154e51cd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:42.815535 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed5ee1-f4cf-4659-a7e3-6eb154e51cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:42.815672 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed5ee1-f4cf-4659-a7e3-6eb154e51cd9 admin admin] PATH_INFO: `/v3/projects/3177e18c06c04a5b89098b72d8ad19b1/users/0483a08230d3487e88b1b36e7bd19727/roles/0e78444e43c744668178971af4087ca0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:42.817769 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ed5ee1-f4cf-4659-a7e3-6eb154e51cd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3177e18c06c04a5b89098b72d8ad19b1, user_id=0483a08230d3487e88b1b36e7bd19727, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:42.820772 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ed5ee1-f4cf-4659-a7e3-6eb154e51cd9 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:42.834180 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1053/2123] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:52:42 2022] PUT /identity/v3/projects/3177e18c06c04a5b89098b72d8ad19b1/users/0483a08230d3487e88b1b36e7bd19727/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:42.845946 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7a145-41f4-4a2a-827f-230746a2264e None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:42.846399 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7a145-41f4-4a2a-827f-230746a2264e None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:42.846791 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7a145-41f4-4a2a-827f-230746a2264e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:42.886023 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-35d7a145-41f4-4a2a-827f-230746a2264e None None] MFA Rules not processed for user `0483a08230d3487e88b1b36e7bd19727`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:42.887378 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-35d7a145-41f4-4a2a-827f-230746a2264e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:42.989292 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1071/2124] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:42 2022] POST /identity/v3/auth/tokens => generated 3293 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:42.997638 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ed5ee1-f4cf-4659-a7e3-6eb154e51cd9 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:43.064397 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ed5ee1-f4cf-4659-a7e3-6eb154e51cd9 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:43.069321 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90052e-c593-41c9-ba89-1496f9730dbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:43.070011 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90052e-c593-41c9-ba89-1496f9730dbe service cinder] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:43.073698 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90052e-c593-41c9-ba89-1496f9730dbe service cinder] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:43.074147 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90052e-c593-41c9-ba89-1496f9730dbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:43.085250 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e90052e-c593-41c9-ba89-1496f9730dbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:43.086695 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e90052e-c593-41c9-ba89-1496f9730dbe service cinder] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:43.097359 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1054/2125] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:52:42 2022] GET /identity/v3/auth/tokens => generated 3293 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:52:43.181977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d7a145-41f4-4a2a-827f-230746a2264e None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:43.201462 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d7a145-41f4-4a2a-827f-230746a2264e None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:43.201462 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10201b7c-b45e-4e13-ac41-cbb2523b9e4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:43.201462 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10201b7c-b45e-4e13-ac41-cbb2523b9e4d service cinder] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:43.201462 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10201b7c-b45e-4e13-ac41-cbb2523b9e4d service cinder] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:43.201839 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10201b7c-b45e-4e13-ac41-cbb2523b9e4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:43.232147 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:43.268578 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10201b7c-b45e-4e13-ac41-cbb2523b9e4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:43.270040 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10201b7c-b45e-4e13-ac41-cbb2523b9e4d service cinder] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:43.279791 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1072/2126] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:52:43 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:52:44.599854 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90052e-c593-41c9-ba89-1496f9730dbe service cinder] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:44.601306 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-6e90052e-c593-41c9-ba89-1496f9730dbe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:44.664802 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:44.683922 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90052e-c593-41c9-ba89-1496f9730dbe service cinder] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:44.688772 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a67ba37-688c-4e09-9920-e8b7dcb97ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:44.689558 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a67ba37-688c-4e09-9920-e8b7dcb97ebf admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:44.689935 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a67ba37-688c-4e09-9920-e8b7dcb97ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:44.690311 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a67ba37-688c-4e09-9920-e8b7dcb97ebf admin admin] PATH_INFO: `/v3/users/46360a6ed1384bf8bfc6be98659509f6` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:44.691600 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a67ba37-688c-4e09-9920-e8b7dcb97ebf admin admin] RBAC: Authorizing `identity:delete_user(user_id=46360a6ed1384bf8bfc6be98659509f6)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:44.694217 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a67ba37-688c-4e09-9920-e8b7dcb97ebf admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:44.746914 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-9a67ba37-688c-4e09-9920-e8b7dcb97ebf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46360a6ed1384bf8bfc6be98659509f6'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:44.747595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-9a67ba37-688c-4e09-9920-e8b7dcb97ebf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46360a6ed1384bf8bfc6be98659509f6'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:44.748250 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-9a67ba37-688c-4e09-9920-e8b7dcb97ebf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46360a6ed1384bf8bfc6be98659509f6'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:44.762520 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-9a67ba37-688c-4e09-9920-e8b7dcb97ebf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46360a6ed1384bf8bfc6be98659509f6'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:44.769621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-9a67ba37-688c-4e09-9920-e8b7dcb97ebf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46360a6ed1384bf8bfc6be98659509f6'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:44.790231 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1055/2127] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:52:44 2022] DELETE /identity/v3/users/46360a6ed1384bf8bfc6be98659509f6 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:44.792217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd6f5a-ae02-48d9-a01b-3dc4527c4c8f None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:44.793282 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd6f5a-ae02-48d9-a01b-3dc4527c4c8f None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:44.793788 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd6f5a-ae02-48d9-a01b-3dc4527c4c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:44.815212 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-e0bd6f5a-ae02-48d9-a01b-3dc4527c4c8f None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:44.816490 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-e0bd6f5a-ae02-48d9-a01b-3dc4527c4c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:44.895846 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1073/2128] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:44.903619 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a67ba37-688c-4e09-9920-e8b7dcb97ebf admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:44.905325 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-9a67ba37-688c-4e09-9920-e8b7dcb97ebf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:44.963904 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a67ba37-688c-4e09-9920-e8b7dcb97ebf admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:44.967526 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aa33e9-5be7-46f1-b15d-7b45b8a098a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:44.968163 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa33e9-5be7-46f1-b15d-7b45b8a098a3 service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:44.968443 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa33e9-5be7-46f1-b15d-7b45b8a098a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:44.968718 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa33e9-5be7-46f1-b15d-7b45b8a098a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:44.978036 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aa33e9-5be7-46f1-b15d-7b45b8a098a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:44.979474 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aa33e9-5be7-46f1-b15d-7b45b8a098a3 service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:44.989106 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1056/2129] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:52:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:52:45.352858 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bd6f5a-ae02-48d9-a01b-3dc4527c4c8f None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:45.354534 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-e0bd6f5a-ae02-48d9-a01b-3dc4527c4c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:45.372181 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bd6f5a-ae02-48d9-a01b-3dc4527c4c8f None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:45.377938 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff8b1e-74b1-4b2e-9f81-43cc55942c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:45.378588 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff8b1e-74b1-4b2e-9f81-43cc55942c65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:45.378871 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff8b1e-74b1-4b2e-9f81-43cc55942c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:45.379146 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff8b1e-74b1-4b2e-9f81-43cc55942c65 admin admin] PATH_INFO: `/v3/projects/2ea32124f33d45c1a667dd6ece8bfb99` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:45.381376 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff8b1e-74b1-4b2e-9f81-43cc55942c65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ea32124f33d45c1a667dd6ece8bfb99)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:45.383367 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff8b1e-74b1-4b2e-9f81-43cc55942c65 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:45.432789 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-d5ff8b1e-74b1-4b2e-9f81-43cc55942c65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ea32124f33d45c1a667dd6ece8bfb99', 'request_id': 'req-d5ff8b1e-74b1-4b2e-9f81-43cc55942c65'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:45.439192 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-d5ff8b1e-74b1-4b2e-9f81-43cc55942c65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ea32124f33d45c1a667dd6ece8bfb99', 'request_id': 'req-d5ff8b1e-74b1-4b2e-9f81-43cc55942c65'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:45.447373 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-d5ff8b1e-74b1-4b2e-9f81-43cc55942c65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ea32124f33d45c1a667dd6ece8bfb99', 'request_id': 'req-d5ff8b1e-74b1-4b2e-9f81-43cc55942c65'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:45.458118 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-d5ff8b1e-74b1-4b2e-9f81-43cc55942c65 admin admin] The token cache is being invalidate because project 2ea32124f33d45c1a667dd6ece8bfb99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:52:45.459051 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-d5ff8b1e-74b1-4b2e-9f81-43cc55942c65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:45.459743 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-d5ff8b1e-74b1-4b2e-9f81-43cc55942c65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:45.461319 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1074/2130] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:52:45 2022] DELETE /identity/v3/projects/2ea32124f33d45c1a667dd6ece8bfb99 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:52:45.465444 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aa33e9-5be7-46f1-b15d-7b45b8a098a3 service neutron] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:45.475450 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aa33e9-5be7-46f1-b15d-7b45b8a098a3 service neutron] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:45.531603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1463f88-7b73-40f0-9e26-759fa867afb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:45.532273 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1463f88-7b73-40f0-9e26-759fa867afb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:45.532553 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1463f88-7b73-40f0-9e26-759fa867afb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:45.533341 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1463f88-7b73-40f0-9e26-759fa867afb0 admin admin] PATH_INFO: `/v3/users/eb8196f17d7949c5a5da0a36b03dc33f` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:45.533994 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1463f88-7b73-40f0-9e26-759fa867afb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb8196f17d7949c5a5da0a36b03dc33f)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:45.535970 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1463f88-7b73-40f0-9e26-759fa867afb0 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:45.549317 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:45.582824 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-d1463f88-7b73-40f0-9e26-759fa867afb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb8196f17d7949c5a5da0a36b03dc33f'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:45.583730 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-d1463f88-7b73-40f0-9e26-759fa867afb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb8196f17d7949c5a5da0a36b03dc33f'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:45.584266 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-d1463f88-7b73-40f0-9e26-759fa867afb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb8196f17d7949c5a5da0a36b03dc33f'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:45.598414 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-d1463f88-7b73-40f0-9e26-759fa867afb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb8196f17d7949c5a5da0a36b03dc33f'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:45.606510 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-d1463f88-7b73-40f0-9e26-759fa867afb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb8196f17d7949c5a5da0a36b03dc33f'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:45.619820 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1057/2131] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:52:45 2022] DELETE /identity/v3/users/eb8196f17d7949c5a5da0a36b03dc33f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:45.898660 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff8b1e-74b1-4b2e-9f81-43cc55942c65 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:45.900931 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-d5ff8b1e-74b1-4b2e-9f81-43cc55942c65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:45.925435 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff8b1e-74b1-4b2e-9f81-43cc55942c65 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:45.976394 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:45.988152 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933cc340-f702-485e-984c-87dc836d262a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:45.988787 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933cc340-f702-485e-984c-87dc836d262a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:45.989084 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933cc340-f702-485e-984c-87dc836d262a admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:45.989846 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933cc340-f702-485e-984c-87dc836d262a admin admin] PATH_INFO: `/v3/projects/ffee4fe12ff44aafafd6e85c092d8b71` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:45.990996 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933cc340-f702-485e-984c-87dc836d262a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffee4fe12ff44aafafd6e85c092d8b71)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:45.992955 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933cc340-f702-485e-984c-87dc836d262a admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:46.039254 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-933cc340-f702-485e-984c-87dc836d262a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffee4fe12ff44aafafd6e85c092d8b71', 'request_id': 'req-933cc340-f702-485e-984c-87dc836d262a'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:46.045389 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-933cc340-f702-485e-984c-87dc836d262a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffee4fe12ff44aafafd6e85c092d8b71', 'request_id': 'req-933cc340-f702-485e-984c-87dc836d262a'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:46.053300 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-933cc340-f702-485e-984c-87dc836d262a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffee4fe12ff44aafafd6e85c092d8b71', 'request_id': 'req-933cc340-f702-485e-984c-87dc836d262a'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:46.063727 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-933cc340-f702-485e-984c-87dc836d262a admin admin] The token cache is being invalidate because project ffee4fe12ff44aafafd6e85c092d8b71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:52:46.064013 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-933cc340-f702-485e-984c-87dc836d262a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:46.064676 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-933cc340-f702-485e-984c-87dc836d262a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:46.066220 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1075/2132] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:52:45 2022] DELETE /identity/v3/projects/ffee4fe12ff44aafafd6e85c092d8b71 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:46.070915 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1463f88-7b73-40f0-9e26-759fa867afb0 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:46.072337 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-d1463f88-7b73-40f0-9e26-759fa867afb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:46.085138 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1463f88-7b73-40f0-9e26-759fa867afb0 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:46.140776 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d8b09-9b65-4b83-985a-cd721c1cba89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:46.141406 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d8b09-9b65-4b83-985a-cd721c1cba89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:46.141680 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d8b09-9b65-4b83-985a-cd721c1cba89 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:46.141949 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d8b09-9b65-4b83-985a-cd721c1cba89 admin admin] PATH_INFO: `/v3/users/dec6296f540645ae9eb873ac6b05a93d` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:46.143084 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35d8b09-9b65-4b83-985a-cd721c1cba89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dec6296f540645ae9eb873ac6b05a93d)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:46.145038 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35d8b09-9b65-4b83-985a-cd721c1cba89 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:46.192665 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-a35d8b09-9b65-4b83-985a-cd721c1cba89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dec6296f540645ae9eb873ac6b05a93d'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:46.194132 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-a35d8b09-9b65-4b83-985a-cd721c1cba89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dec6296f540645ae9eb873ac6b05a93d'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:46.194132 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-a35d8b09-9b65-4b83-985a-cd721c1cba89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dec6296f540645ae9eb873ac6b05a93d'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:46.206070 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-a35d8b09-9b65-4b83-985a-cd721c1cba89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dec6296f540645ae9eb873ac6b05a93d'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:46.213085 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-a35d8b09-9b65-4b83-985a-cd721c1cba89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dec6296f540645ae9eb873ac6b05a93d'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:46.233276 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1058/2133] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:52:46 2022] DELETE /identity/v3/users/dec6296f540645ae9eb873ac6b05a93d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:46.602943 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933cc340-f702-485e-984c-87dc836d262a admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:46.604726 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-933cc340-f702-485e-984c-87dc836d262a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:46.623401 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933cc340-f702-485e-984c-87dc836d262a admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:46.684535 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b1c66c-5d27-4e1d-954a-fc9c38058310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:46.685173 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1c66c-5d27-4e1d-954a-fc9c38058310 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:46.685767 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1c66c-5d27-4e1d-954a-fc9c38058310 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:46.685925 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1c66c-5d27-4e1d-954a-fc9c38058310 admin admin] PATH_INFO: `/v3/projects/8a265396be6f402abd771346e2b47e1f` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:46.687300 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b1c66c-5d27-4e1d-954a-fc9c38058310 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a265396be6f402abd771346e2b47e1f)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:46.689401 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b1c66c-5d27-4e1d-954a-fc9c38058310 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:46.741020 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-89b1c66c-5d27-4e1d-954a-fc9c38058310 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a265396be6f402abd771346e2b47e1f', 'request_id': 'req-89b1c66c-5d27-4e1d-954a-fc9c38058310'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:46.747164 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-89b1c66c-5d27-4e1d-954a-fc9c38058310 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a265396be6f402abd771346e2b47e1f', 'request_id': 'req-89b1c66c-5d27-4e1d-954a-fc9c38058310'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:46.756002 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-89b1c66c-5d27-4e1d-954a-fc9c38058310 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a265396be6f402abd771346e2b47e1f', 'request_id': 'req-89b1c66c-5d27-4e1d-954a-fc9c38058310'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:46.766448 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-89b1c66c-5d27-4e1d-954a-fc9c38058310 admin admin] The token cache is being invalidate because project 8a265396be6f402abd771346e2b47e1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:52:46.766765 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-89b1c66c-5d27-4e1d-954a-fc9c38058310 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:46.767428 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-89b1c66c-5d27-4e1d-954a-fc9c38058310 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:46.769089 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1076/2134] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:52:46 2022] DELETE /identity/v3/projects/8a265396be6f402abd771346e2b47e1f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:46.774506 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef10a95-918d-439b-b32a-e611590814a3 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:46.774951 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef10a95-918d-439b-b32a-e611590814a3 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:46.775042 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef10a95-918d-439b-b32a-e611590814a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:46.797169 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-eef10a95-918d-439b-b32a-e611590814a3 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:46.799067 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-eef10a95-918d-439b-b32a-e611590814a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:46.805669 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:46.864178 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1059/2135] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:46.873170 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90de8bd-cf50-4d57-8dff-116ad4354d05 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:46.873454 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90de8bd-cf50-4d57-8dff-116ad4354d05 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:46.873981 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90de8bd-cf50-4d57-8dff-116ad4354d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:46.893803 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-b90de8bd-cf50-4d57-8dff-116ad4354d05 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:46.895004 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-b90de8bd-cf50-4d57-8dff-116ad4354d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:46.903597 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1077/2136] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:46.907936 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef10a95-918d-439b-b32a-e611590814a3 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:46.918381 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef10a95-918d-439b-b32a-e611590814a3 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:46.923236 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2cef02-1659-4b25-a721-f4fd2c58e44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:46.923888 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2cef02-1659-4b25-a721-f4fd2c58e44d admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:46.924169 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2cef02-1659-4b25-a721-f4fd2c58e44d admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:46.924443 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2cef02-1659-4b25-a721-f4fd2c58e44d admin admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:46.925164 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2cef02-1659-4b25-a721-f4fd2c58e44d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:52:46.925467 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2cef02-1659-4b25-a721-f4fd2c58e44d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:46.927131 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2cef02-1659-4b25-a721-f4fd2c58e44d admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:46.939506 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1060/2137] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:52:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:52:46.944501 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90de8bd-cf50-4d57-8dff-116ad4354d05 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:46.950289 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:46.954292 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90de8bd-cf50-4d57-8dff-116ad4354d05 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:46.960657 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18415ce7-cb97-42d0-9f88-b1ecfc907862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:46.961412 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18415ce7-cb97-42d0-9f88-b1ecfc907862 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:46.961840 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18415ce7-cb97-42d0-9f88-b1ecfc907862 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:46.962153 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18415ce7-cb97-42d0-9f88-b1ecfc907862 admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:46.962990 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18415ce7-cb97-42d0-9f88-b1ecfc907862 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:46.964930 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18415ce7-cb97-42d0-9f88-b1ecfc907862 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:46.992193 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1078/2138] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:52:46 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:52:46.996436 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2cef02-1659-4b25-a721-f4fd2c58e44d admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:47.009880 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2cef02-1659-4b25-a721-f4fd2c58e44d admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:47.072399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085b057-4516-41e1-bbb0-3e95eb84490e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:47.073010 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085b057-4516-41e1-bbb0-3e95eb84490e admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:47.073291 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085b057-4516-41e1-bbb0-3e95eb84490e admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:47.073564 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085b057-4516-41e1-bbb0-3e95eb84490e admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:47.074232 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3085b057-4516-41e1-bbb0-3e95eb84490e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:47.076188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3085b057-4516-41e1-bbb0-3e95eb84490e admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:47.097753 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1061/2139] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:52:46 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:52:47.101901 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18415ce7-cb97-42d0-9f88-b1ecfc907862 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:47.112080 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18415ce7-cb97-42d0-9f88-b1ecfc907862 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:47.116727 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615352e9-1f01-4389-a55a-7cf6dd8d9fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:47.117263 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615352e9-1f01-4389-a55a-7cf6dd8d9fae admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:47.117540 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615352e9-1f01-4389-a55a-7cf6dd8d9fae admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:47.117809 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615352e9-1f01-4389-a55a-7cf6dd8d9fae admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:47.118499 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615352e9-1f01-4389-a55a-7cf6dd8d9fae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:47.120133 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615352e9-1f01-4389-a55a-7cf6dd8d9fae admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:47.130784 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1079/2140] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:52:47.134583 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085b057-4516-41e1-bbb0-3e95eb84490e admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:47.144635 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085b057-4516-41e1-bbb0-3e95eb84490e admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:47.149682 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765c470c-8ba2-4174-ab5c-c7aa3a7b46fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:47.150334 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c470c-8ba2-4174-ab5c-c7aa3a7b46fa admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:47.150610 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c470c-8ba2-4174-ab5c-c7aa3a7b46fa admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:47.150879 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c470c-8ba2-4174-ab5c-c7aa3a7b46fa admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:47.151572 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765c470c-8ba2-4174-ab5c-c7aa3a7b46fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:47.153238 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765c470c-8ba2-4174-ab5c-c7aa3a7b46fa admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:47.165109 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1062/2141] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:52:47.169252 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615352e9-1f01-4389-a55a-7cf6dd8d9fae admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:47.179252 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615352e9-1f01-4389-a55a-7cf6dd8d9fae admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:47.183891 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b9238-fd90-4538-b597-7a3172e125ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:47.184546 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b9238-fd90-4538-b597-7a3172e125ca admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:47.184823 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b9238-fd90-4538-b597-7a3172e125ca admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:47.185087 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b9238-fd90-4538-b597-7a3172e125ca admin admin] PATH_INFO: `/v3/projects/852096232da14dbca191be862498d745/users/d104404f7fa549da84792f694d68ebec/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:47.201115 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1b9238-fd90-4538-b597-7a3172e125ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=852096232da14dbca191be862498d745, user_id=d104404f7fa549da84792f694d68ebec, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:47.203833 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1b9238-fd90-4538-b597-7a3172e125ca admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:47.216999 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1080/2142] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:52:47 2022] PUT /identity/v3/projects/852096232da14dbca191be862498d745/users/d104404f7fa549da84792f694d68ebec/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:47.285697 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e5fcd-dd5a-41bd-9901-e227f6a98813 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:47.285983 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e5fcd-dd5a-41bd-9901-e227f6a98813 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:47.286284 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e5fcd-dd5a-41bd-9901-e227f6a98813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:47.313876 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-688e5fcd-dd5a-41bd-9901-e227f6a98813 None None] MFA Rules not processed for user `d104404f7fa549da84792f694d68ebec`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:47.315086 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-688e5fcd-dd5a-41bd-9901-e227f6a98813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:47.384465 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1063/2143] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:47.395781 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b9238-fd90-4538-b597-7a3172e125ca admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:47.397446 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-ee1b9238-fd90-4538-b597-7a3172e125ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:47.455543 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b9238-fd90-4538-b597-7a3172e125ca admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:47.460184 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8d80cc-b313-40f9-bafc-8637ffd8a8c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:47.460869 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d80cc-b313-40f9-bafc-8637ffd8a8c6 service cinder] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:47.461150 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d80cc-b313-40f9-bafc-8637ffd8a8c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:47.461416 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d80cc-b313-40f9-bafc-8637ffd8a8c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:47.470272 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8d80cc-b313-40f9-bafc-8637ffd8a8c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:47.471732 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8d80cc-b313-40f9-bafc-8637ffd8a8c6 service cinder] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:47.480940 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1081/2144] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:52:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:52:56.435128 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688e5fcd-dd5a-41bd-9901-e227f6a98813 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:56.446725 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:56.509690 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688e5fcd-dd5a-41bd-9901-e227f6a98813 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:56.515681 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71f921-ce6c-412d-98c0-7b6dc77d8a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:56.515681 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71f921-ce6c-412d-98c0-7b6dc77d8a4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:56.515930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71f921-ce6c-412d-98c0-7b6dc77d8a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:56.517641 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71f921-ce6c-412d-98c0-7b6dc77d8a4b admin admin] PATH_INFO: `/v3/users/d104404f7fa549da84792f694d68ebec` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:56.519328 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e71f921-ce6c-412d-98c0-7b6dc77d8a4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d104404f7fa549da84792f694d68ebec)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:56.521891 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e71f921-ce6c-412d-98c0-7b6dc77d8a4b admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:56.582586 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-8e71f921-ce6c-412d-98c0-7b6dc77d8a4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd104404f7fa549da84792f694d68ebec'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:56.582925 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-8e71f921-ce6c-412d-98c0-7b6dc77d8a4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd104404f7fa549da84792f694d68ebec'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:56.583794 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-8e71f921-ce6c-412d-98c0-7b6dc77d8a4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd104404f7fa549da84792f694d68ebec'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:56.596197 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-8e71f921-ce6c-412d-98c0-7b6dc77d8a4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd104404f7fa549da84792f694d68ebec'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:56.604298 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-8e71f921-ce6c-412d-98c0-7b6dc77d8a4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd104404f7fa549da84792f694d68ebec'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:56.618074 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1064/2145] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:52:56 2022] DELETE /identity/v3/users/d104404f7fa549da84792f694d68ebec => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:56.623617 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e02c3f0-03db-4c4b-b7b8-d17ad35254ee None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:56.624362 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e02c3f0-03db-4c4b-b7b8-d17ad35254ee None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:56.624609 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e02c3f0-03db-4c4b-b7b8-d17ad35254ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:56.646340 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-3e02c3f0-03db-4c4b-b7b8-d17ad35254ee None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:56.647622 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-3e02c3f0-03db-4c4b-b7b8-d17ad35254ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:56.712693 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1082/2146] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:56.722633 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71f921-ce6c-412d-98c0-7b6dc77d8a4b admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:56.724638 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-8e71f921-ce6c-412d-98c0-7b6dc77d8a4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:56.786867 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71f921-ce6c-412d-98c0-7b6dc77d8a4b admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:56.790692 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8540814-168e-4eae-bdff-a88f55c2ad9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:56.791313 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8540814-168e-4eae-bdff-a88f55c2ad9e service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:56.791597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8540814-168e-4eae-bdff-a88f55c2ad9e service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:56.791938 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8540814-168e-4eae-bdff-a88f55c2ad9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:56.804991 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8540814-168e-4eae-bdff-a88f55c2ad9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:56.807377 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8540814-168e-4eae-bdff-a88f55c2ad9e service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:56.817770 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1065/2147] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:52:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:52:57.230116 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e02c3f0-03db-4c4b-b7b8-d17ad35254ee None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:57.232196 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-3e02c3f0-03db-4c4b-b7b8-d17ad35254ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:57.254306 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e02c3f0-03db-4c4b-b7b8-d17ad35254ee None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:57.276219 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bab8ccd-061f-4a3e-8c8d-7ec363a8b856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:57.277224 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab8ccd-061f-4a3e-8c8d-7ec363a8b856 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:57.277597 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab8ccd-061f-4a3e-8c8d-7ec363a8b856 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:57.277971 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab8ccd-061f-4a3e-8c8d-7ec363a8b856 admin admin] PATH_INFO: `/v3/projects/852096232da14dbca191be862498d745` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:57.279643 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bab8ccd-061f-4a3e-8c8d-7ec363a8b856 admin admin] RBAC: Authorizing `identity:delete_project(project_id=852096232da14dbca191be862498d745)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:57.281886 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bab8ccd-061f-4a3e-8c8d-7ec363a8b856 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:57.343568 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:57.345282 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-3bab8ccd-061f-4a3e-8c8d-7ec363a8b856 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '852096232da14dbca191be862498d745', 'request_id': 'req-3bab8ccd-061f-4a3e-8c8d-7ec363a8b856'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:57.354602 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-3bab8ccd-061f-4a3e-8c8d-7ec363a8b856 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '852096232da14dbca191be862498d745', 'request_id': 'req-3bab8ccd-061f-4a3e-8c8d-7ec363a8b856'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:57.364982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-3bab8ccd-061f-4a3e-8c8d-7ec363a8b856 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '852096232da14dbca191be862498d745', 'request_id': 'req-3bab8ccd-061f-4a3e-8c8d-7ec363a8b856'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:57.377021 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-3bab8ccd-061f-4a3e-8c8d-7ec363a8b856 admin admin] The token cache is being invalidate because project 852096232da14dbca191be862498d745 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:52:57.377312 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-3bab8ccd-061f-4a3e-8c8d-7ec363a8b856 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:57.377955 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-3bab8ccd-061f-4a3e-8c8d-7ec363a8b856 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:57.379484 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1083/2148] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:52:57 2022] DELETE /identity/v3/projects/852096232da14dbca191be862498d745 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:59.101072 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8540814-168e-4eae-bdff-a88f55c2ad9e service neutron] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:59.102655 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-a8540814-168e-4eae-bdff-a88f55c2ad9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:59.117190 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8540814-168e-4eae-bdff-a88f55c2ad9e service neutron] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:59.146247 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:52:59.187028 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a2885-a7e6-4575-925c-93901d53eb2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:59.187813 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a2885-a7e6-4575-925c-93901d53eb2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:59.188103 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a2885-a7e6-4575-925c-93901d53eb2d admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:59.188438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a2885-a7e6-4575-925c-93901d53eb2d admin admin] PATH_INFO: `/v3/users/2e72882a22074cf980dc72c427cb39f9` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:59.189813 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58a2885-a7e6-4575-925c-93901d53eb2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e72882a22074cf980dc72c427cb39f9)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:59.191995 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58a2885-a7e6-4575-925c-93901d53eb2d admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:59.239208 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-b58a2885-a7e6-4575-925c-93901d53eb2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e72882a22074cf980dc72c427cb39f9'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:59.239908 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-b58a2885-a7e6-4575-925c-93901d53eb2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e72882a22074cf980dc72c427cb39f9'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:59.240521 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-b58a2885-a7e6-4575-925c-93901d53eb2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e72882a22074cf980dc72c427cb39f9'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:59.252946 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-b58a2885-a7e6-4575-925c-93901d53eb2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e72882a22074cf980dc72c427cb39f9'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:59.260273 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-b58a2885-a7e6-4575-925c-93901d53eb2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e72882a22074cf980dc72c427cb39f9'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:59.273801 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1066/2149] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:52:59 2022] DELETE /identity/v3/users/2e72882a22074cf980dc72c427cb39f9 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:59.278417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba4402-35b1-4103-ab44-30d9a1e8b3bd None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:59.278900 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba4402-35b1-4103-ab44-30d9a1e8b3bd None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:59.278993 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba4402-35b1-4103-ab44-30d9a1e8b3bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:59.299215 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-51ba4402-35b1-4103-ab44-30d9a1e8b3bd None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:52:59.300514 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-51ba4402-35b1-4103-ab44-30d9a1e8b3bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:59.369832 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1084/2150] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:52:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:52:59.379870 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a2885-a7e6-4575-925c-93901d53eb2d admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:59.381453 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-b58a2885-a7e6-4575-925c-93901d53eb2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:59.441528 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a2885-a7e6-4575-925c-93901d53eb2d admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:59.445153 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8056d-80b7-4e9b-87e6-d2e0f508c715 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:59.445760 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8056d-80b7-4e9b-87e6-d2e0f508c715 service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:59.446039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8056d-80b7-4e9b-87e6-d2e0f508c715 service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:59.446311 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8056d-80b7-4e9b-87e6-d2e0f508c715 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:59.455782 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc8056d-80b7-4e9b-87e6-d2e0f508c715 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:59.457229 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc8056d-80b7-4e9b-87e6-d2e0f508c715 service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:59.467933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1067/2151] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:52:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:52:59.840294 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba4402-35b1-4103-ab44-30d9a1e8b3bd None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:59.841833 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-51ba4402-35b1-4103-ab44-30d9a1e8b3bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:52:59.856568 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba4402-35b1-4103-ab44-30d9a1e8b3bd None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:52:59.861975 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb82ec-263f-4a85-9fb1-956089af5f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:52:59.863206 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb82ec-263f-4a85-9fb1-956089af5f41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:52:59.863757 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb82ec-263f-4a85-9fb1-956089af5f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:52:59.864064 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb82ec-263f-4a85-9fb1-956089af5f41 admin admin] PATH_INFO: `/v3/projects/b609a950c1014a87a80b0cdf93338340` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:52:59.865511 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afb82ec-263f-4a85-9fb1-956089af5f41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b609a950c1014a87a80b0cdf93338340)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:52:59.867198 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afb82ec-263f-4a85-9fb1-956089af5f41 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:52:59.922329 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-4afb82ec-263f-4a85-9fb1-956089af5f41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b609a950c1014a87a80b0cdf93338340', 'request_id': 'req-4afb82ec-263f-4a85-9fb1-956089af5f41'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:59.928595 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-4afb82ec-263f-4a85-9fb1-956089af5f41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b609a950c1014a87a80b0cdf93338340', 'request_id': 'req-4afb82ec-263f-4a85-9fb1-956089af5f41'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:59.935958 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-4afb82ec-263f-4a85-9fb1-956089af5f41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b609a950c1014a87a80b0cdf93338340', 'request_id': 'req-4afb82ec-263f-4a85-9fb1-956089af5f41'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:59.946978 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-4afb82ec-263f-4a85-9fb1-956089af5f41 admin admin] The token cache is being invalidate because project b609a950c1014a87a80b0cdf93338340 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:52:59.947376 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-4afb82ec-263f-4a85-9fb1-956089af5f41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:59.948054 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-4afb82ec-263f-4a85-9fb1-956089af5f41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:52:59.949587 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1085/2152] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:52:59 2022] DELETE /identity/v3/projects/b609a950c1014a87a80b0cdf93338340 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:52:59.953901 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8056d-80b7-4e9b-87e6-d2e0f508c715 service neutron] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:52:59.965768 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8056d-80b7-4e9b-87e6-d2e0f508c715 service neutron] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:00.015237 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:53:00.027797 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a3fc8-b0ae-4090-984f-92f6505d45e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:00.028458 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a3fc8-b0ae-4090-984f-92f6505d45e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:00.028734 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a3fc8-b0ae-4090-984f-92f6505d45e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:00.029004 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a3fc8-b0ae-4090-984f-92f6505d45e9 admin admin] PATH_INFO: `/v3/users/0483a08230d3487e88b1b36e7bd19727` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:00.030152 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a3fc8-b0ae-4090-984f-92f6505d45e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0483a08230d3487e88b1b36e7bd19727)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:00.032159 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a3fc8-b0ae-4090-984f-92f6505d45e9 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:00.078290 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-1b6a3fc8-b0ae-4090-984f-92f6505d45e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0483a08230d3487e88b1b36e7bd19727'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:00.078960 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-1b6a3fc8-b0ae-4090-984f-92f6505d45e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0483a08230d3487e88b1b36e7bd19727'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:00.079576 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-1b6a3fc8-b0ae-4090-984f-92f6505d45e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0483a08230d3487e88b1b36e7bd19727'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:00.091863 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-1b6a3fc8-b0ae-4090-984f-92f6505d45e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0483a08230d3487e88b1b36e7bd19727'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:00.099337 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-1b6a3fc8-b0ae-4090-984f-92f6505d45e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0483a08230d3487e88b1b36e7bd19727'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:00.112572 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1068/2153] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:52:59 2022] DELETE /identity/v3/users/0483a08230d3487e88b1b36e7bd19727 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:53:00.397267 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb82ec-263f-4a85-9fb1-956089af5f41 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:00.398745 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-4afb82ec-263f-4a85-9fb1-956089af5f41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:00.412304 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb82ec-263f-4a85-9fb1-956089af5f41 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:00.468265 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:53:00.475173 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc99aeec-1ca3-4d93-a615-bd5f16709ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:00.475885 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99aeec-1ca3-4d93-a615-bd5f16709ace admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:00.476520 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99aeec-1ca3-4d93-a615-bd5f16709ace admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:00.478084 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99aeec-1ca3-4d93-a615-bd5f16709ace admin admin] PATH_INFO: `/v3/projects/3177e18c06c04a5b89098b72d8ad19b1` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:00.479481 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc99aeec-1ca3-4d93-a615-bd5f16709ace admin admin] RBAC: Authorizing `identity:delete_project(project_id=3177e18c06c04a5b89098b72d8ad19b1)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:00.481883 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc99aeec-1ca3-4d93-a615-bd5f16709ace admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:00.528307 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-bc99aeec-1ca3-4d93-a615-bd5f16709ace admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3177e18c06c04a5b89098b72d8ad19b1', 'request_id': 'req-bc99aeec-1ca3-4d93-a615-bd5f16709ace'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:00.534394 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-bc99aeec-1ca3-4d93-a615-bd5f16709ace admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3177e18c06c04a5b89098b72d8ad19b1', 'request_id': 'req-bc99aeec-1ca3-4d93-a615-bd5f16709ace'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:00.541770 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-bc99aeec-1ca3-4d93-a615-bd5f16709ace admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3177e18c06c04a5b89098b72d8ad19b1', 'request_id': 'req-bc99aeec-1ca3-4d93-a615-bd5f16709ace'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:00.552594 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-bc99aeec-1ca3-4d93-a615-bd5f16709ace admin admin] The token cache is being invalidate because project 3177e18c06c04a5b89098b72d8ad19b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:53:00.552880 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-bc99aeec-1ca3-4d93-a615-bd5f16709ace admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:00.553531 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-bc99aeec-1ca3-4d93-a615-bd5f16709ace admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:00.555158 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1086/2154] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:53:00 2022] DELETE /identity/v3/projects/3177e18c06c04a5b89098b72d8ad19b1 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:53:00.560108 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854ce7f-a4f4-4de1-94d8-21d351d5cbba None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:00.560446 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854ce7f-a4f4-4de1-94d8-21d351d5cbba None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:00.560719 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854ce7f-a4f4-4de1-94d8-21d351d5cbba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:00.580415 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-d854ce7f-a4f4-4de1-94d8-21d351d5cbba None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:00.581612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-d854ce7f-a4f4-4de1-94d8-21d351d5cbba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:00.641359 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1069/2155] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:53:00.649734 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40334b-ffbe-4fec-8946-881ca14bcb35 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:00.650023 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40334b-ffbe-4fec-8946-881ca14bcb35 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:00.650302 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40334b-ffbe-4fec-8946-881ca14bcb35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:00.669699 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-aa40334b-ffbe-4fec-8946-881ca14bcb35 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:00.670894 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-aa40334b-ffbe-4fec-8946-881ca14bcb35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:00.678944 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1087/2156] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:53:00.683130 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d854ce7f-a4f4-4de1-94d8-21d351d5cbba None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:00.693332 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d854ce7f-a4f4-4de1-94d8-21d351d5cbba None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:00.697950 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cbf8cb-bd1d-42cc-8811-4aa8ac44306b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:00.698552 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cbf8cb-bd1d-42cc-8811-4aa8ac44306b admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:00.698827 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cbf8cb-bd1d-42cc-8811-4aa8ac44306b admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:00.699099 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cbf8cb-bd1d-42cc-8811-4aa8ac44306b admin admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:00.699827 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cbf8cb-bd1d-42cc-8811-4aa8ac44306b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:53:00.700132 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cbf8cb-bd1d-42cc-8811-4aa8ac44306b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:00.701770 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cbf8cb-bd1d-42cc-8811-4aa8ac44306b admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:00.714518 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1070/2157] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:53:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:53:00.719084 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa40334b-ffbe-4fec-8946-881ca14bcb35 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:00.728834 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa40334b-ffbe-4fec-8946-881ca14bcb35 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:00.733568 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46185b07-93c4-4add-9bed-743f6323c41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:00.734183 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46185b07-93c4-4add-9bed-743f6323c41f admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:00.734458 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46185b07-93c4-4add-9bed-743f6323c41f admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:00.734727 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46185b07-93c4-4add-9bed-743f6323c41f admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:00.735403 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46185b07-93c4-4add-9bed-743f6323c41f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:00.737394 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46185b07-93c4-4add-9bed-743f6323c41f admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:00.774586 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1088/2158] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:53:00 2022] POST /identity/v3/projects => generated 367 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:53:00.777128 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cbf8cb-bd1d-42cc-8811-4aa8ac44306b admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:00.788022 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cbf8cb-bd1d-42cc-8811-4aa8ac44306b admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:00.845010 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:53:00.854009 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1242e856-2320-4731-86a2-1425d9c7bd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:00.854912 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242e856-2320-4731-86a2-1425d9c7bd2a admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:00.855562 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242e856-2320-4731-86a2-1425d9c7bd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:00.856236 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242e856-2320-4731-86a2-1425d9c7bd2a admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:00.857191 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1242e856-2320-4731-86a2-1425d9c7bd2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:00.859283 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1242e856-2320-4731-86a2-1425d9c7bd2a admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:00.883744 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1071/2159] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:53:00 2022] POST /identity/v3/users => generated 284 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:53:00.888162 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46185b07-93c4-4add-9bed-743f6323c41f admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:00.899200 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46185b07-93c4-4add-9bed-743f6323c41f admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:00.903973 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c15182-92f1-4a88-9c63-b5aac77134b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:00.904595 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c15182-92f1-4a88-9c63-b5aac77134b3 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:00.904950 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c15182-92f1-4a88-9c63-b5aac77134b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:00.905375 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c15182-92f1-4a88-9c63-b5aac77134b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:00.906258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c15182-92f1-4a88-9c63-b5aac77134b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:00.907912 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c15182-92f1-4a88-9c63-b5aac77134b3 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:00.918785 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1089/2160] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:53:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 12:53:00.922863 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1242e856-2320-4731-86a2-1425d9c7bd2a admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:00.933530 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1242e856-2320-4731-86a2-1425d9c7bd2a admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:00.938374 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d163f4ff-2a57-4e9e-9c5f-512169fb3837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:00.938973 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163f4ff-2a57-4e9e-9c5f-512169fb3837 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:00.939262 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163f4ff-2a57-4e9e-9c5f-512169fb3837 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:00.939538 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163f4ff-2a57-4e9e-9c5f-512169fb3837 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:00.940273 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d163f4ff-2a57-4e9e-9c5f-512169fb3837 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:00.941902 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d163f4ff-2a57-4e9e-9c5f-512169fb3837 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:00.954584 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1072/2161] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:53:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:53:00.958720 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c15182-92f1-4a88-9c63-b5aac77134b3 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:00.968696 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c15182-92f1-4a88-9c63-b5aac77134b3 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:00.973362 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791a35a9-5119-406a-8d96-d187324ca369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:00.974013 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a35a9-5119-406a-8d96-d187324ca369 admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:00.974294 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a35a9-5119-406a-8d96-d187324ca369 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:00.974567 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a35a9-5119-406a-8d96-d187324ca369 admin admin] PATH_INFO: `/v3/projects/0164fadd49004a1f80dfa721e41e1fa2/users/e3755b73ed8047a4b13f880f21a27a7a/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:00.988910 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791a35a9-5119-406a-8d96-d187324ca369 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0164fadd49004a1f80dfa721e41e1fa2, user_id=e3755b73ed8047a4b13f880f21a27a7a, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:00.992157 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791a35a9-5119-406a-8d96-d187324ca369 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:01.007284 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1090/2162] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:53:00 2022] PUT /identity/v3/projects/0164fadd49004a1f80dfa721e41e1fa2/users/e3755b73ed8047a4b13f880f21a27a7a/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:53:01.019243 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36098725-0586-4480-ba7f-1796104a66cf None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:01.019547 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36098725-0586-4480-ba7f-1796104a66cf None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:01.019832 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36098725-0586-4480-ba7f-1796104a66cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:01.047375 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-36098725-0586-4480-ba7f-1796104a66cf None None] MFA Rules not processed for user `e3755b73ed8047a4b13f880f21a27a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:01.048787 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-36098725-0586-4480-ba7f-1796104a66cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:01.113259 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1073/2163] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:01 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:53:01.118304 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791a35a9-5119-406a-8d96-d187324ca369 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:01.181200 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791a35a9-5119-406a-8d96-d187324ca369 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:01.186111 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b367d11-f147-4105-b3dc-4208b28be4da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:01.186731 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b367d11-f147-4105-b3dc-4208b28be4da admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:01.187014 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b367d11-f147-4105-b3dc-4208b28be4da admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:01.187286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b367d11-f147-4105-b3dc-4208b28be4da admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:01.188006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b367d11-f147-4105-b3dc-4208b28be4da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:01.189915 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b367d11-f147-4105-b3dc-4208b28be4da admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:01.220740 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1091/2164] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:53:01 2022] POST /identity/v3/projects => generated 367 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:53:01.221873 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36098725-0586-4480-ba7f-1796104a66cf None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:01.233443 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36098725-0586-4480-ba7f-1796104a66cf None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:01.298231 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4180d37c-a29f-4328-a4a3-31a9ce18450b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:01.298844 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180d37c-a29f-4328-a4a3-31a9ce18450b admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:01.299127 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180d37c-a29f-4328-a4a3-31a9ce18450b admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:01.299398 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180d37c-a29f-4328-a4a3-31a9ce18450b admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:01.300084 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4180d37c-a29f-4328-a4a3-31a9ce18450b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:01.302670 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4180d37c-a29f-4328-a4a3-31a9ce18450b admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:01.324071 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1074/2165] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:53:01 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:53:01.328140 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b367d11-f147-4105-b3dc-4208b28be4da admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:01.338427 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b367d11-f147-4105-b3dc-4208b28be4da admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:01.343096 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c94b19-4397-4d63-848f-6439b9273629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:01.343740 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c94b19-4397-4d63-848f-6439b9273629 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:01.344028 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c94b19-4397-4d63-848f-6439b9273629 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:01.344305 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c94b19-4397-4d63-848f-6439b9273629 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:01.345004 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c94b19-4397-4d63-848f-6439b9273629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:01.346615 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c94b19-4397-4d63-848f-6439b9273629 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:01.349299 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:53:01.360369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1092/2166] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:53:01.364933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4180d37c-a29f-4328-a4a3-31a9ce18450b admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:01.375289 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4180d37c-a29f-4328-a4a3-31a9ce18450b admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:01.379976 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df5550-d0c8-4312-b26e-e32307185f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:01.380624 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df5550-d0c8-4312-b26e-e32307185f49 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:01.380903 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df5550-d0c8-4312-b26e-e32307185f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:01.381177 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df5550-d0c8-4312-b26e-e32307185f49 admin admin] PATH_INFO: `/v3/projects/5a0a3758b9f346bcaec0883426504afe/users/5dfda185e4674ef7a081bdda7d0c0482/roles/0e78444e43c744668178971af4087ca0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:01.399273 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48df5550-d0c8-4312-b26e-e32307185f49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a0a3758b9f346bcaec0883426504afe, user_id=5dfda185e4674ef7a081bdda7d0c0482, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:01.402040 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48df5550-d0c8-4312-b26e-e32307185f49 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:01.415533 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1075/2167] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:53:01 2022] PUT /identity/v3/projects/5a0a3758b9f346bcaec0883426504afe/users/5dfda185e4674ef7a081bdda7d0c0482/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:53:01.419464 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c94b19-4397-4d63-848f-6439b9273629 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:01.429993 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c94b19-4397-4d63-848f-6439b9273629 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:01.487121 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b08b9d5-f4a9-4f97-97af-8a10d136cd30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:01.487783 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08b9d5-f4a9-4f97-97af-8a10d136cd30 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:01.488072 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08b9d5-f4a9-4f97-97af-8a10d136cd30 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:01.488350 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08b9d5-f4a9-4f97-97af-8a10d136cd30 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:01.489081 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b08b9d5-f4a9-4f97-97af-8a10d136cd30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:01.490694 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b08b9d5-f4a9-4f97-97af-8a10d136cd30 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:01.501019 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1093/2168] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:53:01.505191 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df5550-d0c8-4312-b26e-e32307185f49 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:01.515586 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df5550-d0c8-4312-b26e-e32307185f49 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:01.520862 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0e443b-9c60-4945-a280-c71b1b0b9a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:01.521557 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e443b-9c60-4945-a280-c71b1b0b9a7a admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:01.521829 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e443b-9c60-4945-a280-c71b1b0b9a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:01.522083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e443b-9c60-4945-a280-c71b1b0b9a7a admin admin] PATH_INFO: `/v3/projects/5a0a3758b9f346bcaec0883426504afe/users/5dfda185e4674ef7a081bdda7d0c0482/roles/0e78444e43c744668178971af4087ca0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:01.524089 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0e443b-9c60-4945-a280-c71b1b0b9a7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a0a3758b9f346bcaec0883426504afe, user_id=5dfda185e4674ef7a081bdda7d0c0482, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:01.526754 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0e443b-9c60-4945-a280-c71b1b0b9a7a admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:01.537982 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1076/2169] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:53:01 2022] PUT /identity/v3/projects/5a0a3758b9f346bcaec0883426504afe/users/5dfda185e4674ef7a081bdda7d0c0482/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:53:01.548201 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a245f8a4-1949-4456-b584-386b6474f86b None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:01.548493 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a245f8a4-1949-4456-b584-386b6474f86b None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:01.548791 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a245f8a4-1949-4456-b584-386b6474f86b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:01.575547 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-a245f8a4-1949-4456-b584-386b6474f86b None None] MFA Rules not processed for user `5dfda185e4674ef7a081bdda7d0c0482`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:01.576780 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-a245f8a4-1949-4456-b584-386b6474f86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:01.645817 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1094/2170] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:01 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:53:01.657216 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0e443b-9c60-4945-a280-c71b1b0b9a7a admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:01.658956 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-bb0e443b-9c60-4945-a280-c71b1b0b9a7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:01.680642 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:53:01.720113 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0e443b-9c60-4945-a280-c71b1b0b9a7a admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:01.724465 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e075f-11a5-4752-a7b3-31aa2f17f04a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:01.725107 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e075f-11a5-4752-a7b3-31aa2f17f04a service cinder] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:01.725386 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e075f-11a5-4752-a7b3-31aa2f17f04a service cinder] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:01.725663 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e075f-11a5-4752-a7b3-31aa2f17f04a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:01.734710 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282e075f-11a5-4752-a7b3-31aa2f17f04a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:01.736176 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282e075f-11a5-4752-a7b3-31aa2f17f04a service cinder] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:01.745794 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1077/2171] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:53:01 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:53:01.791212 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a245f8a4-1949-4456-b584-386b6474f86b None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:01.792785 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-a245f8a4-1949-4456-b584-386b6474f86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:01.807388 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a245f8a4-1949-4456-b584-386b6474f86b None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:01.811391 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1be1ac-2992-49c6-b936-c729fdfe0641 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:01.812063 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1be1ac-2992-49c6-b936-c729fdfe0641 service cinder] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:01.812391 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1be1ac-2992-49c6-b936-c729fdfe0641 service cinder] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:01.812664 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1be1ac-2992-49c6-b936-c729fdfe0641 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:01.873662 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1be1ac-2992-49c6-b936-c729fdfe0641 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:01.875117 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1be1ac-2992-49c6-b936-c729fdfe0641 service cinder] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:01.885043 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1095/2172] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:53:01 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:53:05.234762 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e075f-11a5-4752-a7b3-31aa2f17f04a service cinder] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:05.296459 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e075f-11a5-4752-a7b3-31aa2f17f04a service cinder] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:05.300987 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77e56b-eafd-42f0-a946-a4c50ac0c7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:05.302123 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77e56b-eafd-42f0-a946-a4c50ac0c7fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:05.302400 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77e56b-eafd-42f0-a946-a4c50ac0c7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:05.302670 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77e56b-eafd-42f0-a946-a4c50ac0c7fb admin admin] PATH_INFO: `/v3/users/e3755b73ed8047a4b13f880f21a27a7a` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:05.303980 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea77e56b-eafd-42f0-a946-a4c50ac0c7fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3755b73ed8047a4b13f880f21a27a7a)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:05.305919 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea77e56b-eafd-42f0-a946-a4c50ac0c7fb admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:05.351492 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-ea77e56b-eafd-42f0-a946-a4c50ac0c7fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3755b73ed8047a4b13f880f21a27a7a'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:05.352169 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-ea77e56b-eafd-42f0-a946-a4c50ac0c7fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3755b73ed8047a4b13f880f21a27a7a'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:05.352776 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-ea77e56b-eafd-42f0-a946-a4c50ac0c7fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3755b73ed8047a4b13f880f21a27a7a'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:05.364637 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-ea77e56b-eafd-42f0-a946-a4c50ac0c7fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3755b73ed8047a4b13f880f21a27a7a'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:05.377909 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-ea77e56b-eafd-42f0-a946-a4c50ac0c7fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3755b73ed8047a4b13f880f21a27a7a'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:05.391033 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1078/2173] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:53:05 2022] DELETE /identity/v3/users/e3755b73ed8047a4b13f880f21a27a7a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:53:05.396119 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd45d2-e83d-4b18-80fe-7631dc651312 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:05.396618 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd45d2-e83d-4b18-80fe-7631dc651312 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:05.396719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd45d2-e83d-4b18-80fe-7631dc651312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:05.416740 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-97dd45d2-e83d-4b18-80fe-7631dc651312 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:05.418019 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-97dd45d2-e83d-4b18-80fe-7631dc651312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:05.431450 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:53:05.487406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1096/2174] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:53:05.497288 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77e56b-eafd-42f0-a946-a4c50ac0c7fb admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:05.509972 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-ea77e56b-eafd-42f0-a946-a4c50ac0c7fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:05.554761 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:53:05.579805 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77e56b-eafd-42f0-a946-a4c50ac0c7fb admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:05.584351 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a1ec2-fbd6-41b3-9735-6ab7c6413a0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:05.585222 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a1ec2-fbd6-41b3-9735-6ab7c6413a0f service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:05.585507 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a1ec2-fbd6-41b3-9735-6ab7c6413a0f service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:05.585842 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a1ec2-fbd6-41b3-9735-6ab7c6413a0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:05.596616 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37a1ec2-fbd6-41b3-9735-6ab7c6413a0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:05.600040 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37a1ec2-fbd6-41b3-9735-6ab7c6413a0f service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:05.610711 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1079/2175] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:53:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:53:05.976446 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd45d2-e83d-4b18-80fe-7631dc651312 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:05.978204 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-97dd45d2-e83d-4b18-80fe-7631dc651312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:05.992402 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd45d2-e83d-4b18-80fe-7631dc651312 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:05.997269 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d0c9d9-d1dd-4c7e-90bc-6101e8bc5498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:05.998005 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d0c9d9-d1dd-4c7e-90bc-6101e8bc5498 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:05.998353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d0c9d9-d1dd-4c7e-90bc-6101e8bc5498 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:05.998723 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d0c9d9-d1dd-4c7e-90bc-6101e8bc5498 admin admin] PATH_INFO: `/v3/projects/0164fadd49004a1f80dfa721e41e1fa2` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:06.000344 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d0c9d9-d1dd-4c7e-90bc-6101e8bc5498 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0164fadd49004a1f80dfa721e41e1fa2)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:06.002556 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d0c9d9-d1dd-4c7e-90bc-6101e8bc5498 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:06.056708 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-d2d0c9d9-d1dd-4c7e-90bc-6101e8bc5498 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0164fadd49004a1f80dfa721e41e1fa2', 'request_id': 'req-d2d0c9d9-d1dd-4c7e-90bc-6101e8bc5498'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:06.064444 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-d2d0c9d9-d1dd-4c7e-90bc-6101e8bc5498 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0164fadd49004a1f80dfa721e41e1fa2', 'request_id': 'req-d2d0c9d9-d1dd-4c7e-90bc-6101e8bc5498'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:06.073591 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-d2d0c9d9-d1dd-4c7e-90bc-6101e8bc5498 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0164fadd49004a1f80dfa721e41e1fa2', 'request_id': 'req-d2d0c9d9-d1dd-4c7e-90bc-6101e8bc5498'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:06.085686 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-d2d0c9d9-d1dd-4c7e-90bc-6101e8bc5498 admin admin] The token cache is being invalidate because project 0164fadd49004a1f80dfa721e41e1fa2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:53:06.085888 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-d2d0c9d9-d1dd-4c7e-90bc-6101e8bc5498 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:06.086594 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-d2d0c9d9-d1dd-4c7e-90bc-6101e8bc5498 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:06.088176 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1097/2176] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:53:05 2022] DELETE /identity/v3/projects/0164fadd49004a1f80dfa721e41e1fa2 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:53:06.092457 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a1ec2-fbd6-41b3-9735-6ab7c6413a0f service neutron] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:06.104343 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a1ec2-fbd6-41b3-9735-6ab7c6413a0f service neutron] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:06.176339 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593342be-0588-4700-bced-a0f8bcbb4b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:06.176339 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593342be-0588-4700-bced-a0f8bcbb4b73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:06.182730 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593342be-0588-4700-bced-a0f8bcbb4b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:06.182730 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593342be-0588-4700-bced-a0f8bcbb4b73 admin admin] PATH_INFO: `/v3/users/5dfda185e4674ef7a081bdda7d0c0482` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:06.185356 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593342be-0588-4700-bced-a0f8bcbb4b73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dfda185e4674ef7a081bdda7d0c0482)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:06.188412 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593342be-0588-4700-bced-a0f8bcbb4b73 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:06.244166 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-593342be-0588-4700-bced-a0f8bcbb4b73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dfda185e4674ef7a081bdda7d0c0482'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:06.244797 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-593342be-0588-4700-bced-a0f8bcbb4b73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dfda185e4674ef7a081bdda7d0c0482'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:06.245387 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-593342be-0588-4700-bced-a0f8bcbb4b73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dfda185e4674ef7a081bdda7d0c0482'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:06.262678 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-593342be-0588-4700-bced-a0f8bcbb4b73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dfda185e4674ef7a081bdda7d0c0482'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:06.269019 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-593342be-0588-4700-bced-a0f8bcbb4b73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dfda185e4674ef7a081bdda7d0c0482'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:06.283620 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1080/2177] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:53:06 2022] DELETE /identity/v3/users/5dfda185e4674ef7a081bdda7d0c0482 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:53:06.569273 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d0c9d9-d1dd-4c7e-90bc-6101e8bc5498 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:06.570775 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-d2d0c9d9-d1dd-4c7e-90bc-6101e8bc5498 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:06.586550 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d0c9d9-d1dd-4c7e-90bc-6101e8bc5498 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:06.648359 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaaf78e-bef9-47c2-9744-e65ba8574f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:06.649022 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaaf78e-bef9-47c2-9744-e65ba8574f0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:06.649340 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaaf78e-bef9-47c2-9744-e65ba8574f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:06.649623 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaaf78e-bef9-47c2-9744-e65ba8574f0d admin admin] PATH_INFO: `/v3/projects/5a0a3758b9f346bcaec0883426504afe` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:06.650840 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaaf78e-bef9-47c2-9744-e65ba8574f0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a0a3758b9f346bcaec0883426504afe)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:06.653014 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaaf78e-bef9-47c2-9744-e65ba8574f0d admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:06.697640 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:53:06.701526 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-9aaaf78e-bef9-47c2-9744-e65ba8574f0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a0a3758b9f346bcaec0883426504afe', 'request_id': 'req-9aaaf78e-bef9-47c2-9744-e65ba8574f0d'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:06.711805 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-9aaaf78e-bef9-47c2-9744-e65ba8574f0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a0a3758b9f346bcaec0883426504afe', 'request_id': 'req-9aaaf78e-bef9-47c2-9744-e65ba8574f0d'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:06.719773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-9aaaf78e-bef9-47c2-9744-e65ba8574f0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a0a3758b9f346bcaec0883426504afe', 'request_id': 'req-9aaaf78e-bef9-47c2-9744-e65ba8574f0d'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:06.734762 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-9aaaf78e-bef9-47c2-9744-e65ba8574f0d admin admin] The token cache is being invalidate because project 5a0a3758b9f346bcaec0883426504afe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:53:06.735062 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-9aaaf78e-bef9-47c2-9744-e65ba8574f0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:06.735791 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-9aaaf78e-bef9-47c2-9744-e65ba8574f0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:06.737552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1098/2178] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:53:06 2022] DELETE /identity/v3/projects/5a0a3758b9f346bcaec0883426504afe => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:53:06.742038 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59475414-b25e-49fd-8082-493a51cad500 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:06.742338 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59475414-b25e-49fd-8082-493a51cad500 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:06.742621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59475414-b25e-49fd-8082-493a51cad500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:06.767287 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-59475414-b25e-49fd-8082-493a51cad500 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:06.768665 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-59475414-b25e-49fd-8082-493a51cad500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:06.838081 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1081/2179] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:53:06.838341 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaaf78e-bef9-47c2-9744-e65ba8574f0d admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:06.839872 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-9aaaf78e-bef9-47c2-9744-e65ba8574f0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:06.845531 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0865bfd2-df7e-42c2-90d9-e8039828060a None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:06.845818 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0865bfd2-df7e-42c2-90d9-e8039828060a None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:06.846094 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0865bfd2-df7e-42c2-90d9-e8039828060a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:06.854614 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:53:06.854926 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaaf78e-bef9-47c2-9744-e65ba8574f0d admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:06.860414 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8219c71-b715-4e2c-a053-911bb9f46e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:06.861133 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8219c71-b715-4e2c-a053-911bb9f46e7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:06.861420 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8219c71-b715-4e2c-a053-911bb9f46e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:06.861699 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8219c71-b715-4e2c-a053-911bb9f46e7b admin admin] PATH_INFO: `/v3/users/e482fc46d40f45f58b295012beb17ba6` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:06.863525 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8219c71-b715-4e2c-a053-911bb9f46e7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e482fc46d40f45f58b295012beb17ba6)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:06.865177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8219c71-b715-4e2c-a053-911bb9f46e7b admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:06.869568 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-0865bfd2-df7e-42c2-90d9-e8039828060a None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:06.871488 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-0865bfd2-df7e-42c2-90d9-e8039828060a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:06.879140 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1082/2180] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:53:06.883507 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0865bfd2-df7e-42c2-90d9-e8039828060a None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:06.893007 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0865bfd2-df7e-42c2-90d9-e8039828060a None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:06.899250 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937adbf0-9c4c-4e18-8441-75464d176bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:06.900059 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937adbf0-9c4c-4e18-8441-75464d176bc3 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:06.900341 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937adbf0-9c4c-4e18-8441-75464d176bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:06.900659 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937adbf0-9c4c-4e18-8441-75464d176bc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:06.901393 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937adbf0-9c4c-4e18-8441-75464d176bc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:53:06.901689 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937adbf0-9c4c-4e18-8441-75464d176bc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:06.903320 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937adbf0-9c4c-4e18-8441-75464d176bc3 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:06.918353 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1083/2181] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:53:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:53:06.919825 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-d8219c71-b715-4e2c-a053-911bb9f46e7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e482fc46d40f45f58b295012beb17ba6'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:06.922636 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937adbf0-9c4c-4e18-8441-75464d176bc3 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:06.928849 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-d8219c71-b715-4e2c-a053-911bb9f46e7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e482fc46d40f45f58b295012beb17ba6'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:06.929784 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-d8219c71-b715-4e2c-a053-911bb9f46e7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e482fc46d40f45f58b295012beb17ba6'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:06.934714 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937adbf0-9c4c-4e18-8441-75464d176bc3 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:06.938095 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-d8219c71-b715-4e2c-a053-911bb9f46e7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e482fc46d40f45f58b295012beb17ba6'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:06.939616 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bfe7ab-046e-4ec3-8116-3a9ba88db042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:06.940259 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfe7ab-046e-4ec3-8116-3a9ba88db042 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:06.940546 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfe7ab-046e-4ec3-8116-3a9ba88db042 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:06.940825 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfe7ab-046e-4ec3-8116-3a9ba88db042 admin admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:06.941490 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bfe7ab-046e-4ec3-8116-3a9ba88db042 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:06.943475 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bfe7ab-046e-4ec3-8116-3a9ba88db042 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:06.950867 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-d8219c71-b715-4e2c-a053-911bb9f46e7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e482fc46d40f45f58b295012beb17ba6'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:06.958362 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1099/2182] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:53:06 2022] DELETE /identity/v3/users/e482fc46d40f45f58b295012beb17ba6 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:53:06.963899 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5487a-e983-4831-8cde-ac61a4e9b038 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:06.964240 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5487a-e983-4831-8cde-ac61a4e9b038 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:06.964578 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5487a-e983-4831-8cde-ac61a4e9b038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:06.977606 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1084/2183] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:53:06 2022] POST /identity/v3/projects => generated 361 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:53:06.979012 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bfe7ab-046e-4ec3-8116-3a9ba88db042 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:06.981546 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-85bfe7ab-046e-4ec3-8116-3a9ba88db042 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:06.989003 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-3da5487a-e983-4831-8cde-ac61a4e9b038 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:06.990141 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-3da5487a-e983-4831-8cde-ac61a4e9b038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:07.049721 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bfe7ab-046e-4ec3-8116-3a9ba88db042 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:07.054579 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf670be-765e-489f-a703-d1a2dc3091d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:07.055202 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf670be-765e-489f-a703-d1a2dc3091d0 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:07.055767 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf670be-765e-489f-a703-d1a2dc3091d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:07.055887 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1100/2184] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:53:07.055992 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf670be-765e-489f-a703-d1a2dc3091d0 admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:07.056506 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf670be-765e-489f-a703-d1a2dc3091d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:07.058732 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf670be-765e-489f-a703-d1a2dc3091d0 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:07.065822 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da5487a-e983-4831-8cde-ac61a4e9b038 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:07.068104 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-3da5487a-e983-4831-8cde-ac61a4e9b038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:07.082070 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1085/2185] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:53:06 2022] POST /identity/v3/users => generated 281 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:53:07.086891 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf670be-765e-489f-a703-d1a2dc3091d0 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:07.097280 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf670be-765e-489f-a703-d1a2dc3091d0 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:07.102674 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b6a0c-9324-4326-bb7c-52430a145fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:07.103295 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b6a0c-9324-4326-bb7c-52430a145fa4 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:07.103582 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b6a0c-9324-4326-bb7c-52430a145fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:07.103888 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b6a0c-9324-4326-bb7c-52430a145fa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:07.104596 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053b6a0c-9324-4326-bb7c-52430a145fa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:07.106238 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053b6a0c-9324-4326-bb7c-52430a145fa4 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:07.117866 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1086/2186] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 12:53:07.121681 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:53:07.122057 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b6a0c-9324-4326-bb7c-52430a145fa4 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:07.133300 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b6a0c-9324-4326-bb7c-52430a145fa4 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:07.135864 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da5487a-e983-4831-8cde-ac61a4e9b038 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:07.138504 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9447a423-51b3-4c45-9c84-e8e5bc55c466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:07.139147 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447a423-51b3-4c45-9c84-e8e5bc55c466 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:07.139432 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447a423-51b3-4c45-9c84-e8e5bc55c466 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:07.139737 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447a423-51b3-4c45-9c84-e8e5bc55c466 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:07.140279 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3440e382-965d-42b0-9371-998766addbb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:07.140436 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9447a423-51b3-4c45-9c84-e8e5bc55c466 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:07.140896 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3440e382-965d-42b0-9371-998766addbb7 service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:07.141178 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3440e382-965d-42b0-9371-998766addbb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:07.141444 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3440e382-965d-42b0-9371-998766addbb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:07.142046 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9447a423-51b3-4c45-9c84-e8e5bc55c466 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:07.151261 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3440e382-965d-42b0-9371-998766addbb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:07.153141 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1087/2187] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 12:53:07.153291 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3440e382-965d-42b0-9371-998766addbb7 service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:07.157106 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9447a423-51b3-4c45-9c84-e8e5bc55c466 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:07.165608 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1101/2188] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:53:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 12:53:07.171501 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9447a423-51b3-4c45-9c84-e8e5bc55c466 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:07.177296 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c28a308-0b37-4e72-ba6e-dd96ee31899e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:07.177965 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28a308-0b37-4e72-ba6e-dd96ee31899e admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:07.178249 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28a308-0b37-4e72-ba6e-dd96ee31899e admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:07.178535 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28a308-0b37-4e72-ba6e-dd96ee31899e admin admin] PATH_INFO: `/v3/projects/273a85368c1143e0b07a2d96c79c35b4/users/6b1baeee499f4a6b8c856b53295b1a8e/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:07.194921 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c28a308-0b37-4e72-ba6e-dd96ee31899e admin admin] RBAC: Authorizing `identity:create_grant(project_id=273a85368c1143e0b07a2d96c79c35b4, user_id=6b1baeee499f4a6b8c856b53295b1a8e, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:07.197788 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c28a308-0b37-4e72-ba6e-dd96ee31899e admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:07.212636 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1088/2189] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:53:07 2022] PUT /identity/v3/projects/273a85368c1143e0b07a2d96c79c35b4/users/6b1baeee499f4a6b8c856b53295b1a8e/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:53:07.224722 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbca0bf-9399-4c24-b19e-5786b421afb6 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:07.225498 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbca0bf-9399-4c24-b19e-5786b421afb6 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:07.225784 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbca0bf-9399-4c24-b19e-5786b421afb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:07.260568 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-3bbca0bf-9399-4c24-b19e-5786b421afb6 None None] MFA Rules not processed for user `6b1baeee499f4a6b8c856b53295b1a8e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:07.261846 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-3bbca0bf-9399-4c24-b19e-5786b421afb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:07.363841 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1102/2190] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:07 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:53:07.397030 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c28a308-0b37-4e72-ba6e-dd96ee31899e admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:07.398771 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-2c28a308-0b37-4e72-ba6e-dd96ee31899e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:07.480121 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c28a308-0b37-4e72-ba6e-dd96ee31899e admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:07.487672 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449d1b4-4cd4-4c23-8250-33dfdd208036 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:07.488683 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449d1b4-4cd4-4c23-8250-33dfdd208036 service cinder] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:07.491136 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449d1b4-4cd4-4c23-8250-33dfdd208036 service cinder] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:07.491136 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449d1b4-4cd4-4c23-8250-33dfdd208036 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:07.504125 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3449d1b4-4cd4-4c23-8250-33dfdd208036 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:07.506247 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3449d1b4-4cd4-4c23-8250-33dfdd208036 service cinder] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:07.513703 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbca0bf-9399-4c24-b19e-5786b421afb6 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:07.515472 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-3bbca0bf-9399-4c24-b19e-5786b421afb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:07.527248 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1089/2191] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:53:07 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:53:07.544519 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449d1b4-4cd4-4c23-8250-33dfdd208036 service cinder] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:07.555417 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:53:07.630299 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbca0bf-9399-4c24-b19e-5786b421afb6 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:07.635104 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23132484-9a23-4338-80af-44becccfba9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:07.636027 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23132484-9a23-4338-80af-44becccfba9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:07.636602 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23132484-9a23-4338-80af-44becccfba9c admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:07.636979 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23132484-9a23-4338-80af-44becccfba9c admin admin] PATH_INFO: `/v3/projects/32882d8e5b2445dabf15c6d081077b0f` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:07.639273 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23132484-9a23-4338-80af-44becccfba9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=32882d8e5b2445dabf15c6d081077b0f)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:07.641932 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23132484-9a23-4338-80af-44becccfba9c admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:07.643160 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449d1b4-4cd4-4c23-8250-33dfdd208036 service cinder] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:07.649545 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc0948-121b-4eee-84a5-042ca5b88a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:07.650204 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc0948-121b-4eee-84a5-042ca5b88a2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:07.650461 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc0948-121b-4eee-84a5-042ca5b88a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:07.650869 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc0948-121b-4eee-84a5-042ca5b88a2b admin admin] PATH_INFO: `/v3/users/6b1baeee499f4a6b8c856b53295b1a8e` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:07.652192 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fc0948-121b-4eee-84a5-042ca5b88a2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b1baeee499f4a6b8c856b53295b1a8e)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:07.654262 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fc0948-121b-4eee-84a5-042ca5b88a2b admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:07.705727 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-23132484-9a23-4338-80af-44becccfba9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32882d8e5b2445dabf15c6d081077b0f', 'request_id': 'req-23132484-9a23-4338-80af-44becccfba9c'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:07.714290 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-23132484-9a23-4338-80af-44becccfba9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32882d8e5b2445dabf15c6d081077b0f', 'request_id': 'req-23132484-9a23-4338-80af-44becccfba9c'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:07.724526 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-23132484-9a23-4338-80af-44becccfba9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32882d8e5b2445dabf15c6d081077b0f', 'request_id': 'req-23132484-9a23-4338-80af-44becccfba9c'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:07.732346 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-68fc0948-121b-4eee-84a5-042ca5b88a2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b1baeee499f4a6b8c856b53295b1a8e'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:07.733347 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-68fc0948-121b-4eee-84a5-042ca5b88a2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b1baeee499f4a6b8c856b53295b1a8e'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:07.734211 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-68fc0948-121b-4eee-84a5-042ca5b88a2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b1baeee499f4a6b8c856b53295b1a8e'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:07.744511 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-23132484-9a23-4338-80af-44becccfba9c admin admin] The token cache is being invalidate because project 32882d8e5b2445dabf15c6d081077b0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:53:07.745145 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-23132484-9a23-4338-80af-44becccfba9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:07.746107 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-23132484-9a23-4338-80af-44becccfba9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:07.748006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1103/2192] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:53:07 2022] DELETE /identity/v3/projects/32882d8e5b2445dabf15c6d081077b0f => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:53:07.756467 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-68fc0948-121b-4eee-84a5-042ca5b88a2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b1baeee499f4a6b8c856b53295b1a8e'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:07.757229 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442825e-15e6-4c59-942b-11843bdf9812 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:07.757807 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442825e-15e6-4c59-942b-11843bdf9812 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:07.758681 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442825e-15e6-4c59-942b-11843bdf9812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:07.763695 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-68fc0948-121b-4eee-84a5-042ca5b88a2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b1baeee499f4a6b8c856b53295b1a8e'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:07.777914 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:53:07.779982 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1090/2193] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:53:07 2022] DELETE /identity/v3/users/6b1baeee499f4a6b8c856b53295b1a8e => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:53:07.786545 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-7442825e-15e6-4c59-942b-11843bdf9812 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:07.786910 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad929013-cd72-438b-ac1e-901cb8141085 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:07.787218 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad929013-cd72-438b-ac1e-901cb8141085 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:07.787684 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad929013-cd72-438b-ac1e-901cb8141085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:07.788710 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-7442825e-15e6-4c59-942b-11843bdf9812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:07.809886 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-ad929013-cd72-438b-ac1e-901cb8141085 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:07.811662 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-ad929013-cd72-438b-ac1e-901cb8141085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:07.884710 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1104/2194] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:53:07.892532 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83078b7d-f752-4f30-8da0-3f6acfcb92da None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:07.893321 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83078b7d-f752-4f30-8da0-3f6acfcb92da None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:07.893774 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83078b7d-f752-4f30-8da0-3f6acfcb92da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:07.909193 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1091/2195] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:53:07.919087 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad929013-cd72-438b-ac1e-901cb8141085 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:07.920802 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-ad929013-cd72-438b-ac1e-901cb8141085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:07.921958 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-83078b7d-f752-4f30-8da0-3f6acfcb92da None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:07.923145 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-83078b7d-f752-4f30-8da0-3f6acfcb92da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:07.932202 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1105/2196] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:53:07.936102 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83078b7d-f752-4f30-8da0-3f6acfcb92da None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:07.945684 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83078b7d-f752-4f30-8da0-3f6acfcb92da None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:07.950491 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f51ae5c-4bbe-438e-bb9e-043d61bd7298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:07.951098 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f51ae5c-4bbe-438e-bb9e-043d61bd7298 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:07.951377 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f51ae5c-4bbe-438e-bb9e-043d61bd7298 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:07.951677 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f51ae5c-4bbe-438e-bb9e-043d61bd7298 admin admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:07.952381 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f51ae5c-4bbe-438e-bb9e-043d61bd7298 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:53:07.952681 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f51ae5c-4bbe-438e-bb9e-043d61bd7298 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:07.954285 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f51ae5c-4bbe-438e-bb9e-043d61bd7298 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:07.962153 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:53:07.969950 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1106/2197] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:53:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:53:07.973942 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f51ae5c-4bbe-438e-bb9e-043d61bd7298 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:07.985818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f51ae5c-4bbe-438e-bb9e-043d61bd7298 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:07.990850 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc832b19-c32f-4164-b736-4936dacc2573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:07.991460 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc832b19-c32f-4164-b736-4936dacc2573 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:07.991776 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc832b19-c32f-4164-b736-4936dacc2573 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:07.992047 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc832b19-c32f-4164-b736-4936dacc2573 admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:07.992702 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc832b19-c32f-4164-b736-4936dacc2573 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:07.993702 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad929013-cd72-438b-ac1e-901cb8141085 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:07.994614 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc832b19-c32f-4164-b736-4936dacc2573 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:07.998646 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b24aed-4843-4a14-bd37-f8c093d94d59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:08.000016 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b24aed-4843-4a14-bd37-f8c093d94d59 service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:08.000016 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b24aed-4843-4a14-bd37-f8c093d94d59 service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:08.000300 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b24aed-4843-4a14-bd37-f8c093d94d59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:08.011997 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b24aed-4843-4a14-bd37-f8c093d94d59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:08.013528 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b24aed-4843-4a14-bd37-f8c093d94d59 service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:08.024115 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1092/2198] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:53:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 12:53:08.026748 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1107/2199] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:53:07 2022] POST /identity/v3/projects => generated 369 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:53:08.032702 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b24aed-4843-4a14-bd37-f8c093d94d59 service neutron] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:08.042773 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b24aed-4843-4a14-bd37-f8c093d94d59 service neutron] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:08.116177 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2fa681-f850-4d76-8bac-8fde54f3d04a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:08.116895 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2fa681-f850-4d76-8bac-8fde54f3d04a admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:08.117243 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2fa681-f850-4d76-8bac-8fde54f3d04a admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:08.117582 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2fa681-f850-4d76-8bac-8fde54f3d04a admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:08.118319 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2fa681-f850-4d76-8bac-8fde54f3d04a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:08.120868 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2fa681-f850-4d76-8bac-8fde54f3d04a admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:08.154416 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1093/2200] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:53:08 2022] POST /identity/v3/users => generated 285 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:53:08.156742 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc832b19-c32f-4164-b736-4936dacc2573 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:08.169325 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc832b19-c32f-4164-b736-4936dacc2573 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:08.175120 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c61f74-a925-400c-a81c-ff102a79421d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:08.175764 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c61f74-a925-400c-a81c-ff102a79421d admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:08.176050 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c61f74-a925-400c-a81c-ff102a79421d admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:08.176328 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c61f74-a925-400c-a81c-ff102a79421d admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:08.177309 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c61f74-a925-400c-a81c-ff102a79421d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:08.179304 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c61f74-a925-400c-a81c-ff102a79421d admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:08.194184 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1108/2201] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:53:08.199195 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2fa681-f850-4d76-8bac-8fde54f3d04a admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:08.211917 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2fa681-f850-4d76-8bac-8fde54f3d04a admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:08.217117 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adebc78-34e7-43fa-9809-2785842dcf74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:08.217798 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adebc78-34e7-43fa-9809-2785842dcf74 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:08.218083 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adebc78-34e7-43fa-9809-2785842dcf74 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:08.218352 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adebc78-34e7-43fa-9809-2785842dcf74 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:08.219093 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adebc78-34e7-43fa-9809-2785842dcf74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:08.220710 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adebc78-34e7-43fa-9809-2785842dcf74 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:08.231388 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1094/2202] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:53:08.235483 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c61f74-a925-400c-a81c-ff102a79421d admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:08.246336 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c61f74-a925-400c-a81c-ff102a79421d admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:08.251420 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b5ea29-aa0e-4d31-8fe7-72125c983e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:08.252200 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5ea29-aa0e-4d31-8fe7-72125c983e9a admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:08.252590 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5ea29-aa0e-4d31-8fe7-72125c983e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:08.252973 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5ea29-aa0e-4d31-8fe7-72125c983e9a admin admin] PATH_INFO: `/v3/projects/e80027d473534e0ab00fa91b53221cb3/users/7fe15bf80eed482abe1338f107a1992f/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:08.269106 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b5ea29-aa0e-4d31-8fe7-72125c983e9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e80027d473534e0ab00fa91b53221cb3, user_id=7fe15bf80eed482abe1338f107a1992f, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:08.272105 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b5ea29-aa0e-4d31-8fe7-72125c983e9a admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:08.286363 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1109/2203] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:53:08 2022] PUT /identity/v3/projects/e80027d473534e0ab00fa91b53221cb3/users/7fe15bf80eed482abe1338f107a1992f/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:53:08.297800 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19534a64-c1f5-4e1f-a1be-0f997099f4bb None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:08.298089 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19534a64-c1f5-4e1f-a1be-0f997099f4bb None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:08.298436 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19534a64-c1f5-4e1f-a1be-0f997099f4bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:08.328953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-19534a64-c1f5-4e1f-a1be-0f997099f4bb None None] MFA Rules not processed for user `7fe15bf80eed482abe1338f107a1992f`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:08.330295 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-19534a64-c1f5-4e1f-a1be-0f997099f4bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:08.332947 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b5ea29-aa0e-4d31-8fe7-72125c983e9a admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:08.334981 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-e1b5ea29-aa0e-4d31-8fe7-72125c983e9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:08.401623 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1095/2204] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:08 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:53:08.409095 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b5ea29-aa0e-4d31-8fe7-72125c983e9a admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:08.414210 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93920502-a36f-42d8-b3b1-8005bf6ca418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:08.414551 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19534a64-c1f5-4e1f-a1be-0f997099f4bb None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:08.414954 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93920502-a36f-42d8-b3b1-8005bf6ca418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:08.415206 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93920502-a36f-42d8-b3b1-8005bf6ca418 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:08.415485 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93920502-a36f-42d8-b3b1-8005bf6ca418 admin admin] PATH_INFO: `/v3/projects/273a85368c1143e0b07a2d96c79c35b4` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:08.416545 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-19534a64-c1f5-4e1f-a1be-0f997099f4bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:08.417417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93920502-a36f-42d8-b3b1-8005bf6ca418 admin admin] RBAC: Authorizing `identity:delete_project(project_id=273a85368c1143e0b07a2d96c79c35b4)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:08.419347 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93920502-a36f-42d8-b3b1-8005bf6ca418 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:08.455209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:53:08.459836 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:53:08.470218 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-93920502-a36f-42d8-b3b1-8005bf6ca418 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '273a85368c1143e0b07a2d96c79c35b4', 'request_id': 'req-93920502-a36f-42d8-b3b1-8005bf6ca418'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:08.476081 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19534a64-c1f5-4e1f-a1be-0f997099f4bb None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:08.476613 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-93920502-a36f-42d8-b3b1-8005bf6ca418 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '273a85368c1143e0b07a2d96c79c35b4', 'request_id': 'req-93920502-a36f-42d8-b3b1-8005bf6ca418'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:08.485425 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-93920502-a36f-42d8-b3b1-8005bf6ca418 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '273a85368c1143e0b07a2d96c79c35b4', 'request_id': 'req-93920502-a36f-42d8-b3b1-8005bf6ca418'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:08.496862 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-93920502-a36f-42d8-b3b1-8005bf6ca418 admin admin] The token cache is being invalidate because project 273a85368c1143e0b07a2d96c79c35b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:53:08.497155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-93920502-a36f-42d8-b3b1-8005bf6ca418 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:08.498133 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-93920502-a36f-42d8-b3b1-8005bf6ca418 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:08.499978 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1110/2205] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:53:08 2022] DELETE /identity/v3/projects/273a85368c1143e0b07a2d96c79c35b4 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:53:08.505232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c70884-51af-41d7-9c94-022526c65bab None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:08.505631 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c70884-51af-41d7-9c94-022526c65bab None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:08.505917 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c70884-51af-41d7-9c94-022526c65bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:08.525827 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-c9c70884-51af-41d7-9c94-022526c65bab None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:08.527001 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-c9c70884-51af-41d7-9c94-022526c65bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:08.528677 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50ca9c-65b8-4914-8935-c8a9c0438cd0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2b5853838dc40db89c46038c0606107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:08.529339 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50ca9c-65b8-4914-8935-c8a9c0438cd0 service glance] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:08.529593 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50ca9c-65b8-4914-8935-c8a9c0438cd0 service glance] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:08.529853 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50ca9c-65b8-4914-8935-c8a9c0438cd0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:08.531809 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-2b50ca9c-65b8-4914-8935-c8a9c0438cd0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:08.588200 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1111/2206] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:53:08.592962 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b50ca9c-65b8-4914-8935-c8a9c0438cd0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:08.594415 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b50ca9c-65b8-4914-8935-c8a9c0438cd0 service glance] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:08.596483 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df1402-6fa8-4a19-acc9-237434460fad None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:08.596834 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df1402-6fa8-4a19-acc9-237434460fad None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:08.597124 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df1402-6fa8-4a19-acc9-237434460fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:08.604197 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1096/2207] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:53:08 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 12:53:08.619278 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-c9df1402-6fa8-4a19-acc9-237434460fad None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:08.620542 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-c9df1402-6fa8-4a19-acc9-237434460fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:08.629555 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1112/2208] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:53:08.634016 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50ca9c-65b8-4914-8935-c8a9c0438cd0 service glance] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:08.647157 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50ca9c-65b8-4914-8935-c8a9c0438cd0 service glance] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:08.652030 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fedfe0-e6f7-4f11-9e34-138339112e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:08.652650 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fedfe0-e6f7-4f11-9e34-138339112e91 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:08.652935 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fedfe0-e6f7-4f11-9e34-138339112e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:08.653330 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fedfe0-e6f7-4f11-9e34-138339112e91 admin admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:08.653989 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fedfe0-e6f7-4f11-9e34-138339112e91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:53:08.654299 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fedfe0-e6f7-4f11-9e34-138339112e91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:08.655927 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fedfe0-e6f7-4f11-9e34-138339112e91 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:08.670133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1097/2209] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:53:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:53:08.674449 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9df1402-6fa8-4a19-acc9-237434460fad None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:08.684681 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9df1402-6fa8-4a19-acc9-237434460fad None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:08.690091 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e182649e-be68-4263-8ce8-01f69e7b5ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:08.690741 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182649e-be68-4263-8ce8-01f69e7b5ef4 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:08.690883 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182649e-be68-4263-8ce8-01f69e7b5ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:08.691202 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182649e-be68-4263-8ce8-01f69e7b5ef4 admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:08.692060 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e182649e-be68-4263-8ce8-01f69e7b5ef4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:08.693972 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e182649e-be68-4263-8ce8-01f69e7b5ef4 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:08.726322 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1113/2210] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:53:08 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:53:08.728698 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fedfe0-e6f7-4f11-9e34-138339112e91 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:08.740171 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fedfe0-e6f7-4f11-9e34-138339112e91 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:08.804367 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed7dc9-abdd-421e-80b4-1abd5ba9cc5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:08.805059 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed7dc9-abdd-421e-80b4-1abd5ba9cc5e admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:08.805434 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed7dc9-abdd-421e-80b4-1abd5ba9cc5e admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:08.805897 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed7dc9-abdd-421e-80b4-1abd5ba9cc5e admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:08.806726 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ed7dc9-abdd-421e-80b4-1abd5ba9cc5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:08.808833 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ed7dc9-abdd-421e-80b4-1abd5ba9cc5e admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:08.835364 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1098/2211] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:53:08 2022] POST /identity/v3/users => generated 287 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:53:08.839477 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e182649e-be68-4263-8ce8-01f69e7b5ef4 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:08.850550 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e182649e-be68-4263-8ce8-01f69e7b5ef4 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:08.855407 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ad9b1-8650-420c-9617-b74a84ae7c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:08.856076 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ad9b1-8650-420c-9617-b74a84ae7c37 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:08.856356 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ad9b1-8650-420c-9617-b74a84ae7c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:08.856625 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ad9b1-8650-420c-9617-b74a84ae7c37 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:08.857444 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069ad9b1-8650-420c-9617-b74a84ae7c37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:08.858928 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069ad9b1-8650-420c-9617-b74a84ae7c37 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:08.874022 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1114/2212] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 12:53:08.878112 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed7dc9-abdd-421e-80b4-1abd5ba9cc5e admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:08.891318 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed7dc9-abdd-421e-80b4-1abd5ba9cc5e admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:08.897468 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe3020f-be66-40a7-bb6a-a301fb1c6fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:08.900090 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3020f-be66-40a7-bb6a-a301fb1c6fa2 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:08.900384 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3020f-be66-40a7-bb6a-a301fb1c6fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:08.900665 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3020f-be66-40a7-bb6a-a301fb1c6fa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:08.901394 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe3020f-be66-40a7-bb6a-a301fb1c6fa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:08.903102 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe3020f-be66-40a7-bb6a-a301fb1c6fa2 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:08.923614 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1099/2213] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:53:08.924606 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ad9b1-8650-420c-9617-b74a84ae7c37 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:08.936133 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ad9b1-8650-420c-9617-b74a84ae7c37 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:08.947871 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a3e3fa-47af-477d-9738-572a089d2b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:08.948600 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a3e3fa-47af-477d-9738-572a089d2b6e admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:08.948811 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a3e3fa-47af-477d-9738-572a089d2b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:08.949081 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a3e3fa-47af-477d-9738-572a089d2b6e admin admin] PATH_INFO: `/v3/projects/5f9e190b1a5a423e9cd061e5a465866a/users/ea0e82d1094a42399db90438f5012b44/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:08.981352 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a3e3fa-47af-477d-9738-572a089d2b6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f9e190b1a5a423e9cd061e5a465866a, user_id=ea0e82d1094a42399db90438f5012b44, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:08.984249 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a3e3fa-47af-477d-9738-572a089d2b6e admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:08.998167 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1115/2214] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:53:08 2022] PUT /identity/v3/projects/5f9e190b1a5a423e9cd061e5a465866a/users/ea0e82d1094a42399db90438f5012b44/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:53:09.009397 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ccd48-b593-4ff2-9c4a-e1e14ba5a595 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:09.009724 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ccd48-b593-4ff2-9c4a-e1e14ba5a595 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:09.010094 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ccd48-b593-4ff2-9c4a-e1e14ba5a595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:09.041733 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:53:09.044805 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-0f1ccd48-b593-4ff2-9c4a-e1e14ba5a595 None None] MFA Rules not processed for user `ea0e82d1094a42399db90438f5012b44`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:09.047036 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-0f1ccd48-b593-4ff2-9c4a-e1e14ba5a595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:09.126277 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1100/2215] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:53:09.131005 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a3e3fa-47af-477d-9738-572a089d2b6e admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:09.197613 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:53:09.202436 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a3e3fa-47af-477d-9738-572a089d2b6e admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:09.209412 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73518c2-0cfb-4e13-95bf-b98cbf6f97e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:09.210359 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73518c2-0cfb-4e13-95bf-b98cbf6f97e5 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:09.210820 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73518c2-0cfb-4e13-95bf-b98cbf6f97e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:09.211101 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73518c2-0cfb-4e13-95bf-b98cbf6f97e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:09.211859 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73518c2-0cfb-4e13-95bf-b98cbf6f97e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:09.213879 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73518c2-0cfb-4e13-95bf-b98cbf6f97e5 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:09.240859 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1116/2216] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:53:09 2022] POST /identity/v3/projects => generated 375 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:53:09.245031 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1ccd48-b593-4ff2-9c4a-e1e14ba5a595 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:09.256276 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1ccd48-b593-4ff2-9c4a-e1e14ba5a595 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:09.312229 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe89de5-2c9d-47a0-9ca1-57e8e930173f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:09.312838 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe89de5-2c9d-47a0-9ca1-57e8e930173f admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:09.313131 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe89de5-2c9d-47a0-9ca1-57e8e930173f admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:09.313407 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe89de5-2c9d-47a0-9ca1-57e8e930173f admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:09.314182 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe89de5-2c9d-47a0-9ca1-57e8e930173f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:09.316147 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe89de5-2c9d-47a0-9ca1-57e8e930173f admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:09.338460 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1101/2217] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:53:09 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:53:09.342620 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73518c2-0cfb-4e13-95bf-b98cbf6f97e5 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:09.352993 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73518c2-0cfb-4e13-95bf-b98cbf6f97e5 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:09.357608 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d635ae72-0032-45d4-9b16-70e9bc921970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:09.358213 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635ae72-0032-45d4-9b16-70e9bc921970 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:09.358493 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635ae72-0032-45d4-9b16-70e9bc921970 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:09.358766 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635ae72-0032-45d4-9b16-70e9bc921970 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:09.359456 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d635ae72-0032-45d4-9b16-70e9bc921970 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:09.361112 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d635ae72-0032-45d4-9b16-70e9bc921970 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:09.371524 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1117/2218] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:53:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:53:09.375934 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe89de5-2c9d-47a0-9ca1-57e8e930173f admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:09.386610 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe89de5-2c9d-47a0-9ca1-57e8e930173f admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:09.391165 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a944c1-0cca-4b45-93f6-5c9e71954192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:09.391843 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a944c1-0cca-4b45-93f6-5c9e71954192 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:09.392125 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a944c1-0cca-4b45-93f6-5c9e71954192 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:09.392395 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a944c1-0cca-4b45-93f6-5c9e71954192 admin admin] PATH_INFO: `/v3/projects/09f74f5041f44789ab401d198911f780/users/e493972d160845e2af25ec4bc43c87b1/roles/0e78444e43c744668178971af4087ca0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:09.409151 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a944c1-0cca-4b45-93f6-5c9e71954192 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09f74f5041f44789ab401d198911f780, user_id=e493972d160845e2af25ec4bc43c87b1, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:09.413013 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a944c1-0cca-4b45-93f6-5c9e71954192 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:09.430054 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1102/2219] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:53:09 2022] PUT /identity/v3/projects/09f74f5041f44789ab401d198911f780/users/e493972d160845e2af25ec4bc43c87b1/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:53:09.434098 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d635ae72-0032-45d4-9b16-70e9bc921970 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:09.445102 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d635ae72-0032-45d4-9b16-70e9bc921970 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:09.502636 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca8ecc-b4ec-4aac-b7b0-a387e3543a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:09.503392 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca8ecc-b4ec-4aac-b7b0-a387e3543a84 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:09.503677 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca8ecc-b4ec-4aac-b7b0-a387e3543a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:09.503955 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca8ecc-b4ec-4aac-b7b0-a387e3543a84 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:09.505294 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aca8ecc-b4ec-4aac-b7b0-a387e3543a84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:09.507463 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aca8ecc-b4ec-4aac-b7b0-a387e3543a84 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:09.518269 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1118/2220] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:53:09 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:53:09.523296 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a944c1-0cca-4b45-93f6-5c9e71954192 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:09.535405 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a944c1-0cca-4b45-93f6-5c9e71954192 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:09.541259 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e32c66-eb8c-407d-831f-5b3631691f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:09.541854 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e32c66-eb8c-407d-831f-5b3631691f43 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:09.542132 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e32c66-eb8c-407d-831f-5b3631691f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:09.542406 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e32c66-eb8c-407d-831f-5b3631691f43 admin admin] PATH_INFO: `/v3/projects/09f74f5041f44789ab401d198911f780/users/e493972d160845e2af25ec4bc43c87b1/roles/0e78444e43c744668178971af4087ca0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:09.544377 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e32c66-eb8c-407d-831f-5b3631691f43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09f74f5041f44789ab401d198911f780, user_id=e493972d160845e2af25ec4bc43c87b1, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:09.547197 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e32c66-eb8c-407d-831f-5b3631691f43 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:09.558691 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1103/2221] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:53:09 2022] PUT /identity/v3/projects/09f74f5041f44789ab401d198911f780/users/e493972d160845e2af25ec4bc43c87b1/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:53:09.570419 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b7700d-9d5a-48d4-8e4a-b01a27747122 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:09.570740 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b7700d-9d5a-48d4-8e4a-b01a27747122 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:09.571158 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b7700d-9d5a-48d4-8e4a-b01a27747122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:09.600316 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-32b7700d-9d5a-48d4-8e4a-b01a27747122 None None] MFA Rules not processed for user `e493972d160845e2af25ec4bc43c87b1`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:09.601581 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-32b7700d-9d5a-48d4-8e4a-b01a27747122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:09.673862 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1119/2222] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:09 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:53:09.686258 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e32c66-eb8c-407d-831f-5b3631691f43 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:09.688036 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-e8e32c66-eb8c-407d-831f-5b3631691f43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:09.767221 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e32c66-eb8c-407d-831f-5b3631691f43 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:09.772266 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01a17db-9f2d-4ee4-a198-ac024d90c195 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:09.772994 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01a17db-9f2d-4ee4-a198-ac024d90c195 service cinder] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:09.773308 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01a17db-9f2d-4ee4-a198-ac024d90c195 service cinder] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:09.773661 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01a17db-9f2d-4ee4-a198-ac024d90c195 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:09.786188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:53:09.847649 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01a17db-9f2d-4ee4-a198-ac024d90c195 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:09.849201 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01a17db-9f2d-4ee4-a198-ac024d90c195 service cinder] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:09.859168 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1104/2223] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:53:09 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:53:11.698478 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b7700d-9d5a-48d4-8e4a-b01a27747122 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:11.700050 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-32b7700d-9d5a-48d4-8e4a-b01a27747122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:11.755599 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:53:11.769952 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b7700d-9d5a-48d4-8e4a-b01a27747122 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:11.775098 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceac54d-29c3-46c0-8836-ef0bb83ef550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:11.775757 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceac54d-29c3-46c0-8836-ef0bb83ef550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:11.776039 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceac54d-29c3-46c0-8836-ef0bb83ef550 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:11.776314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceac54d-29c3-46c0-8836-ef0bb83ef550 admin admin] PATH_INFO: `/v3/users/8b1779409bff4e1fa6610bedd641b8d9` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:11.777572 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceac54d-29c3-46c0-8836-ef0bb83ef550 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b1779409bff4e1fa6610bedd641b8d9)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:11.779586 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceac54d-29c3-46c0-8836-ef0bb83ef550 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:11.832125 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-6ceac54d-29c3-46c0-8836-ef0bb83ef550 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b1779409bff4e1fa6610bedd641b8d9'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:11.839522 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-6ceac54d-29c3-46c0-8836-ef0bb83ef550 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b1779409bff4e1fa6610bedd641b8d9'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:11.840179 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-6ceac54d-29c3-46c0-8836-ef0bb83ef550 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b1779409bff4e1fa6610bedd641b8d9'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:11.847478 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-6ceac54d-29c3-46c0-8836-ef0bb83ef550 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b1779409bff4e1fa6610bedd641b8d9'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:11.862211 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-6ceac54d-29c3-46c0-8836-ef0bb83ef550 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b1779409bff4e1fa6610bedd641b8d9'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:11.869582 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1120/2224] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:53:11 2022] DELETE /identity/v3/users/8b1779409bff4e1fa6610bedd641b8d9 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:53:11.875009 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba7139-c726-4737-87e5-38537aa761c5 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:11.875579 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba7139-c726-4737-87e5-38537aa761c5 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:11.875579 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba7139-c726-4737-87e5-38537aa761c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:11.895888 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-09ba7139-c726-4737-87e5-38537aa761c5 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:11.897156 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-09ba7139-c726-4737-87e5-38537aa761c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:11.958916 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1105/2225] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:53:11.968655 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceac54d-29c3-46c0-8836-ef0bb83ef550 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:11.970058 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-6ceac54d-29c3-46c0-8836-ef0bb83ef550 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:12.036546 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceac54d-29c3-46c0-8836-ef0bb83ef550 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:12.040437 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c516ae4-32a6-4309-83e2-2f36071a4a4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:12.041072 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c516ae4-32a6-4309-83e2-2f36071a4a4c service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:12.041381 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c516ae4-32a6-4309-83e2-2f36071a4a4c service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:12.041634 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c516ae4-32a6-4309-83e2-2f36071a4a4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:12.051892 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c516ae4-32a6-4309-83e2-2f36071a4a4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:12.053509 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c516ae4-32a6-4309-83e2-2f36071a4a4c service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:12.066275 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1121/2226] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:53:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:53:12.341320 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba7139-c726-4737-87e5-38537aa761c5 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:12.342794 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-09ba7139-c726-4737-87e5-38537aa761c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:12.355013 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba7139-c726-4737-87e5-38537aa761c5 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:12.360669 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94af35-8d98-457a-95b1-ad2691f102ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:12.361014 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94af35-8d98-457a-95b1-ad2691f102ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:12.361357 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94af35-8d98-457a-95b1-ad2691f102ad admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:12.361698 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94af35-8d98-457a-95b1-ad2691f102ad admin admin] PATH_INFO: `/v3/projects/20bc5fcf58914d7bbfa1d1be1327f4d3` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:12.363848 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f94af35-8d98-457a-95b1-ad2691f102ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=20bc5fcf58914d7bbfa1d1be1327f4d3)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:12.366375 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f94af35-8d98-457a-95b1-ad2691f102ad admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:12.421688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-4f94af35-8d98-457a-95b1-ad2691f102ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20bc5fcf58914d7bbfa1d1be1327f4d3', 'request_id': 'req-4f94af35-8d98-457a-95b1-ad2691f102ad'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:12.439662 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-4f94af35-8d98-457a-95b1-ad2691f102ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20bc5fcf58914d7bbfa1d1be1327f4d3', 'request_id': 'req-4f94af35-8d98-457a-95b1-ad2691f102ad'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:12.440005 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-4f94af35-8d98-457a-95b1-ad2691f102ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20bc5fcf58914d7bbfa1d1be1327f4d3', 'request_id': 'req-4f94af35-8d98-457a-95b1-ad2691f102ad'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:12.452554 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-4f94af35-8d98-457a-95b1-ad2691f102ad admin admin] The token cache is being invalidate because project 20bc5fcf58914d7bbfa1d1be1327f4d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:53:12.455351 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-4f94af35-8d98-457a-95b1-ad2691f102ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:12.455351 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-4f94af35-8d98-457a-95b1-ad2691f102ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:12.455351 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1106/2227] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:53:12 2022] DELETE /identity/v3/projects/20bc5fcf58914d7bbfa1d1be1327f4d3 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:53:12.460485 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6e775-6083-4592-87ea-667a603e96bc None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:12.460769 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6e775-6083-4592-87ea-667a603e96bc None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:12.461039 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6e775-6083-4592-87ea-667a603e96bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:12.481323 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-8cf6e775-6083-4592-87ea-667a603e96bc None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:12.482523 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-8cf6e775-6083-4592-87ea-667a603e96bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:12.546601 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1122/2228] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:53:12.553981 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0603c8b-b906-4650-807d-1a8f5b994bed None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:12.554274 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0603c8b-b906-4650-807d-1a8f5b994bed None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:12.554578 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0603c8b-b906-4650-807d-1a8f5b994bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:12.580700 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-a0603c8b-b906-4650-807d-1a8f5b994bed None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:12.581924 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-a0603c8b-b906-4650-807d-1a8f5b994bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:12.590942 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1107/2229] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:53:12.595941 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf6e775-6083-4592-87ea-667a603e96bc None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:12.606112 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf6e775-6083-4592-87ea-667a603e96bc None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:12.611917 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0603c8b-b906-4650-807d-1a8f5b994bed None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:12.613372 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-a0603c8b-b906-4650-807d-1a8f5b994bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:12.615145 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c459f5a-dbac-4ca8-9188-890f0679099c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:12.615868 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c459f5a-dbac-4ca8-9188-890f0679099c admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:12.616152 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c459f5a-dbac-4ca8-9188-890f0679099c admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:12.616470 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c459f5a-dbac-4ca8-9188-890f0679099c admin admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:12.617254 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c459f5a-dbac-4ca8-9188-890f0679099c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:53:12.617558 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c459f5a-dbac-4ca8-9188-890f0679099c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:12.619310 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c459f5a-dbac-4ca8-9188-890f0679099c admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:12.626501 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0603c8b-b906-4650-807d-1a8f5b994bed None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:12.631997 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be0e321-3e68-4bee-81e8-f65975433d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:12.632641 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0e321-3e68-4bee-81e8-f65975433d0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:12.632920 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0e321-3e68-4bee-81e8-f65975433d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:12.633196 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0e321-3e68-4bee-81e8-f65975433d0a admin admin] PATH_INFO: `/v3/users/ea0e82d1094a42399db90438f5012b44` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:12.636930 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1123/2230] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:53:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:53:12.637055 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be0e321-3e68-4bee-81e8-f65975433d0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea0e82d1094a42399db90438f5012b44)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:12.637055 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be0e321-3e68-4bee-81e8-f65975433d0a admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:12.637901 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c459f5a-dbac-4ca8-9188-890f0679099c admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:12.642844 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:53:12.650814 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c459f5a-dbac-4ca8-9188-890f0679099c admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:12.656703 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e0007-ceba-48de-a650-a13a0f7b22b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:12.657301 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e0007-ceba-48de-a650-a13a0f7b22b8 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:12.657578 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e0007-ceba-48de-a650-a13a0f7b22b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:12.657846 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e0007-ceba-48de-a650-a13a0f7b22b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:12.658511 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e0007-ceba-48de-a650-a13a0f7b22b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:12.660724 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e0007-ceba-48de-a650-a13a0f7b22b8 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:12.660846 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:53:12.689861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-4be0e321-3e68-4bee-81e8-f65975433d0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea0e82d1094a42399db90438f5012b44'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:12.690565 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-4be0e321-3e68-4bee-81e8-f65975433d0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea0e82d1094a42399db90438f5012b44'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:12.691192 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-4be0e321-3e68-4bee-81e8-f65975433d0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea0e82d1094a42399db90438f5012b44'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:12.695027 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1124/2231] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:53:12 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:53:12.699500 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e0007-ceba-48de-a650-a13a0f7b22b8 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:12.701727 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-773e0007-ceba-48de-a650-a13a0f7b22b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:12.709509 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-4be0e321-3e68-4bee-81e8-f65975433d0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea0e82d1094a42399db90438f5012b44'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:12.718512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-4be0e321-3e68-4bee-81e8-f65975433d0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea0e82d1094a42399db90438f5012b44'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:12.743905 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1108/2232] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:53:12 2022] DELETE /identity/v3/users/ea0e82d1094a42399db90438f5012b44 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:53:12.748075 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3175c47-13d5-4bc0-a766-9805ad2d4b52 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:12.748364 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3175c47-13d5-4bc0-a766-9805ad2d4b52 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:12.748638 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3175c47-13d5-4bc0-a766-9805ad2d4b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:12.769821 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-a3175c47-13d5-4bc0-a766-9805ad2d4b52 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:12.771574 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-a3175c47-13d5-4bc0-a766-9805ad2d4b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:12.782117 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e0007-ceba-48de-a650-a13a0f7b22b8 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:12.844811 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1109/2233] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:53:12.846039 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf16df-65ef-40ee-9351-e689ebde2dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:12.846670 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf16df-65ef-40ee-9351-e689ebde2dd1 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:12.846953 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf16df-65ef-40ee-9351-e689ebde2dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:12.847226 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf16df-65ef-40ee-9351-e689ebde2dd1 admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:12.848028 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbf16df-65ef-40ee-9351-e689ebde2dd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:12.850074 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbf16df-65ef-40ee-9351-e689ebde2dd1 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:12.854760 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3175c47-13d5-4bc0-a766-9805ad2d4b52 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:12.856680 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-a3175c47-13d5-4bc0-a766-9805ad2d4b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:12.876671 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1125/2234] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:53:12 2022] POST /identity/v3/users => generated 287 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:53:12.881398 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf16df-65ef-40ee-9351-e689ebde2dd1 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:12.898981 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf16df-65ef-40ee-9351-e689ebde2dd1 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:12.904747 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0739c685-0d5c-42c6-aeab-7fafee6677f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:12.905478 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0739c685-0d5c-42c6-aeab-7fafee6677f4 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:12.905854 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0739c685-0d5c-42c6-aeab-7fafee6677f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:12.906270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0739c685-0d5c-42c6-aeab-7fafee6677f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:12.907291 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0739c685-0d5c-42c6-aeab-7fafee6677f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:12.909414 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0739c685-0d5c-42c6-aeab-7fafee6677f4 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:12.921842 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1126/2235] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 12:53:12.926184 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0739c685-0d5c-42c6-aeab-7fafee6677f4 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:12.938482 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3175c47-13d5-4bc0-a766-9805ad2d4b52 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:12.939025 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0739c685-0d5c-42c6-aeab-7fafee6677f4 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:12.943193 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfaa778-9983-467f-be9f-df97cebcff9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:12.943578 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfaa778-9983-467f-be9f-df97cebcff9a service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:12.943884 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfaa778-9983-467f-be9f-df97cebcff9a service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:12.944045 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617db9c3-886d-4119-835d-0f22eb7a3903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:12.944154 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfaa778-9983-467f-be9f-df97cebcff9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:12.944696 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617db9c3-886d-4119-835d-0f22eb7a3903 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:12.944974 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617db9c3-886d-4119-835d-0f22eb7a3903 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:12.945250 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617db9c3-886d-4119-835d-0f22eb7a3903 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:12.945949 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617db9c3-886d-4119-835d-0f22eb7a3903 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:12.947559 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617db9c3-886d-4119-835d-0f22eb7a3903 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:12.954359 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfaa778-9983-467f-be9f-df97cebcff9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:12.955823 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfaa778-9983-467f-be9f-df97cebcff9a service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:12.959231 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1127/2236] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 12:53:12.963272 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617db9c3-886d-4119-835d-0f22eb7a3903 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:12.966201 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1110/2237] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:53:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 12:53:12.973917 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617db9c3-886d-4119-835d-0f22eb7a3903 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:12.978765 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a574748-200f-4da8-92f9-b97bcf7818a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:12.979438 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a574748-200f-4da8-92f9-b97bcf7818a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:12.979751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a574748-200f-4da8-92f9-b97bcf7818a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:12.980027 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a574748-200f-4da8-92f9-b97bcf7818a9 admin admin] PATH_INFO: `/v3/projects/4ab951d5e91f4bf7a23a710cdc244360/users/e1fcebd110924d12a6da707dbf57f277/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:12.996440 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a574748-200f-4da8-92f9-b97bcf7818a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ab951d5e91f4bf7a23a710cdc244360, user_id=e1fcebd110924d12a6da707dbf57f277, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:12.999732 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a574748-200f-4da8-92f9-b97bcf7818a9 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:13.016096 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1128/2238] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:53:12 2022] PUT /identity/v3/projects/4ab951d5e91f4bf7a23a710cdc244360/users/e1fcebd110924d12a6da707dbf57f277/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:53:13.027233 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd6653-0260-4347-ac06-5e16fdcc8b15 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:13.027499 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd6653-0260-4347-ac06-5e16fdcc8b15 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:13.027821 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd6653-0260-4347-ac06-5e16fdcc8b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:13.061280 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-2abd6653-0260-4347-ac06-5e16fdcc8b15 None None] MFA Rules not processed for user `e1fcebd110924d12a6da707dbf57f277`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:13.062525 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-2abd6653-0260-4347-ac06-5e16fdcc8b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:13.097731 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:53:13.145823 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1111/2239] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:53:13.156574 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a574748-200f-4da8-92f9-b97bcf7818a9 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:13.158699 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-4a574748-200f-4da8-92f9-b97bcf7818a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:13.207549 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:53:13.232967 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a574748-200f-4da8-92f9-b97bcf7818a9 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:13.237639 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67647b22-3861-4a41-9c40-90fdc85d4c2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:13.238292 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67647b22-3861-4a41-9c40-90fdc85d4c2c service cinder] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:13.238543 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67647b22-3861-4a41-9c40-90fdc85d4c2c service cinder] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:13.238884 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67647b22-3861-4a41-9c40-90fdc85d4c2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:13.248927 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67647b22-3861-4a41-9c40-90fdc85d4c2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:13.250773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67647b22-3861-4a41-9c40-90fdc85d4c2c service cinder] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:13.260513 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1129/2240] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:53:13 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:53:13.261420 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abd6653-0260-4347-ac06-5e16fdcc8b15 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:13.263012 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-2abd6653-0260-4347-ac06-5e16fdcc8b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:13.352559 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abd6653-0260-4347-ac06-5e16fdcc8b15 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:13.357961 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a5984-8d5f-46cf-acd6-87375f533810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:13.358609 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a5984-8d5f-46cf-acd6-87375f533810 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:13.358887 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a5984-8d5f-46cf-acd6-87375f533810 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:13.359197 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a5984-8d5f-46cf-acd6-87375f533810 admin admin] PATH_INFO: `/v3/projects/5f9e190b1a5a423e9cd061e5a465866a` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:13.360466 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a5984-8d5f-46cf-acd6-87375f533810 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f9e190b1a5a423e9cd061e5a465866a)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:13.362390 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a5984-8d5f-46cf-acd6-87375f533810 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:13.416644 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-183a5984-8d5f-46cf-acd6-87375f533810 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f9e190b1a5a423e9cd061e5a465866a', 'request_id': 'req-183a5984-8d5f-46cf-acd6-87375f533810'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:13.426169 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-183a5984-8d5f-46cf-acd6-87375f533810 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f9e190b1a5a423e9cd061e5a465866a', 'request_id': 'req-183a5984-8d5f-46cf-acd6-87375f533810'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:13.437158 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-183a5984-8d5f-46cf-acd6-87375f533810 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f9e190b1a5a423e9cd061e5a465866a', 'request_id': 'req-183a5984-8d5f-46cf-acd6-87375f533810'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:13.450705 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-183a5984-8d5f-46cf-acd6-87375f533810 admin admin] The token cache is being invalidate because project 5f9e190b1a5a423e9cd061e5a465866a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:53:13.451187 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-183a5984-8d5f-46cf-acd6-87375f533810 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:13.451971 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-183a5984-8d5f-46cf-acd6-87375f533810 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:13.453828 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1112/2241] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:53:13 2022] DELETE /identity/v3/projects/5f9e190b1a5a423e9cd061e5a465866a => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:53:13.458685 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67647b22-3861-4a41-9c40-90fdc85d4c2c service cinder] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:13.470280 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67647b22-3861-4a41-9c40-90fdc85d4c2c service cinder] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:13.544066 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8092bb1-f57f-4b5a-ba8b-dcb38a5f1200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:13.544206 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8092bb1-f57f-4b5a-ba8b-dcb38a5f1200 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:13.544470 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8092bb1-f57f-4b5a-ba8b-dcb38a5f1200 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:13.544749 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8092bb1-f57f-4b5a-ba8b-dcb38a5f1200 admin admin] PATH_INFO: `/v3/users/e493972d160845e2af25ec4bc43c87b1` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:13.546071 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8092bb1-f57f-4b5a-ba8b-dcb38a5f1200 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e493972d160845e2af25ec4bc43c87b1)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:13.548322 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8092bb1-f57f-4b5a-ba8b-dcb38a5f1200 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:13.612946 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-f8092bb1-f57f-4b5a-ba8b-dcb38a5f1200 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e493972d160845e2af25ec4bc43c87b1'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:13.621862 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-f8092bb1-f57f-4b5a-ba8b-dcb38a5f1200 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e493972d160845e2af25ec4bc43c87b1'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:13.622666 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-f8092bb1-f57f-4b5a-ba8b-dcb38a5f1200 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e493972d160845e2af25ec4bc43c87b1'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:13.631854 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-f8092bb1-f57f-4b5a-ba8b-dcb38a5f1200 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e493972d160845e2af25ec4bc43c87b1'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:13.645701 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-f8092bb1-f57f-4b5a-ba8b-dcb38a5f1200 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e493972d160845e2af25ec4bc43c87b1'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:13.652082 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1130/2242] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:53:13 2022] DELETE /identity/v3/users/e493972d160845e2af25ec4bc43c87b1 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:53:13.948583 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a5984-8d5f-46cf-acd6-87375f533810 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:13.950105 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-183a5984-8d5f-46cf-acd6-87375f533810 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:13.969320 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a5984-8d5f-46cf-acd6-87375f533810 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:14.028188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941782f6-1209-4466-8722-13d02c52f813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:14.028859 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941782f6-1209-4466-8722-13d02c52f813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:14.029145 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941782f6-1209-4466-8722-13d02c52f813 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:14.029423 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941782f6-1209-4466-8722-13d02c52f813 admin admin] PATH_INFO: `/v3/projects/09f74f5041f44789ab401d198911f780` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:14.030691 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941782f6-1209-4466-8722-13d02c52f813 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09f74f5041f44789ab401d198911f780)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:14.032703 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941782f6-1209-4466-8722-13d02c52f813 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:14.034973 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:53:14.083006 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-941782f6-1209-4466-8722-13d02c52f813 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09f74f5041f44789ab401d198911f780', 'request_id': 'req-941782f6-1209-4466-8722-13d02c52f813'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:14.089206 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-941782f6-1209-4466-8722-13d02c52f813 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09f74f5041f44789ab401d198911f780', 'request_id': 'req-941782f6-1209-4466-8722-13d02c52f813'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:14.096575 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-941782f6-1209-4466-8722-13d02c52f813 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09f74f5041f44789ab401d198911f780', 'request_id': 'req-941782f6-1209-4466-8722-13d02c52f813'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:14.107546 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-941782f6-1209-4466-8722-13d02c52f813 admin admin] The token cache is being invalidate because project 09f74f5041f44789ab401d198911f780 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:53:14.107937 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-941782f6-1209-4466-8722-13d02c52f813 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:14.108557 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-941782f6-1209-4466-8722-13d02c52f813 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:14.110059 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1113/2243] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:53:13 2022] DELETE /identity/v3/projects/09f74f5041f44789ab401d198911f780 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:53:14.115686 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e15ff-ce30-4075-be4f-c70ce94c71fc None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:14.116183 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e15ff-ce30-4075-be4f-c70ce94c71fc None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:14.116298 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e15ff-ce30-4075-be4f-c70ce94c71fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:14.136481 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-ff2e15ff-ce30-4075-be4f-c70ce94c71fc None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:14.137692 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-ff2e15ff-ce30-4075-be4f-c70ce94c71fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:14.193615 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:53:14.209483 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1131/2244] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:53:14.217714 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8fa3a1-43de-47fe-8bd5-6e08cf742c90 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:14.218002 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8fa3a1-43de-47fe-8bd5-6e08cf742c90 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:14.218307 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8fa3a1-43de-47fe-8bd5-6e08cf742c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:14.240818 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-ce8fa3a1-43de-47fe-8bd5-6e08cf742c90 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:14.241964 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-ce8fa3a1-43de-47fe-8bd5-6e08cf742c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:14.250233 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1114/2245] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:53:14.254727 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e15ff-ce30-4075-be4f-c70ce94c71fc None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:14.264840 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e15ff-ce30-4075-be4f-c70ce94c71fc None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:14.269439 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e6f46-5948-4d56-b467-0c31d929260c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:14.270643 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e6f46-5948-4d56-b467-0c31d929260c admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:14.270928 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e6f46-5948-4d56-b467-0c31d929260c admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:14.271197 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e6f46-5948-4d56-b467-0c31d929260c admin admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:14.271918 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345e6f46-5948-4d56-b467-0c31d929260c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:53:14.272219 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345e6f46-5948-4d56-b467-0c31d929260c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:14.273824 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345e6f46-5948-4d56-b467-0c31d929260c admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:14.285669 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1132/2246] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:53:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:53:14.290554 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8fa3a1-43de-47fe-8bd5-6e08cf742c90 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:14.300028 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8fa3a1-43de-47fe-8bd5-6e08cf742c90 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:14.304751 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7789d437-45a2-4bda-a2a8-b542d6f4f3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:14.305348 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789d437-45a2-4bda-a2a8-b542d6f4f3ac admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:14.305630 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789d437-45a2-4bda-a2a8-b542d6f4f3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:14.305902 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789d437-45a2-4bda-a2a8-b542d6f4f3ac admin admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:14.306560 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7789d437-45a2-4bda-a2a8-b542d6f4f3ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:14.308513 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7789d437-45a2-4bda-a2a8-b542d6f4f3ac admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:14.340366 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1115/2247] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:53:14 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:53:14.341593 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e6f46-5948-4d56-b467-0c31d929260c admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:14.352770 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e6f46-5948-4d56-b467-0c31d929260c admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:14.408473 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84338d5-fc97-47bf-85dc-d216f56b7b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:14.409082 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84338d5-fc97-47bf-85dc-d216f56b7b21 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:14.409361 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84338d5-fc97-47bf-85dc-d216f56b7b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:14.409637 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84338d5-fc97-47bf-85dc-d216f56b7b21 admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:14.410287 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84338d5-fc97-47bf-85dc-d216f56b7b21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:14.412842 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84338d5-fc97-47bf-85dc-d216f56b7b21 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:14.433588 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1133/2248] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:53:14 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:53:14.437907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7789d437-45a2-4bda-a2a8-b542d6f4f3ac admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:14.448408 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7789d437-45a2-4bda-a2a8-b542d6f4f3ac admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:14.453071 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a901e0-9f87-4216-9587-341c7379c258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:14.453675 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a901e0-9f87-4216-9587-341c7379c258 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:14.453959 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a901e0-9f87-4216-9587-341c7379c258 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:14.454236 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a901e0-9f87-4216-9587-341c7379c258 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:14.454926 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a901e0-9f87-4216-9587-341c7379c258 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:14.456563 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a901e0-9f87-4216-9587-341c7379c258 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:14.466906 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1116/2249] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:53:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 12:53:14.470750 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84338d5-fc97-47bf-85dc-d216f56b7b21 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:14.482118 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84338d5-fc97-47bf-85dc-d216f56b7b21 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:14.492555 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb9875-573c-4364-84f5-f2d1ae8f5404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:14.493087 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb9875-573c-4364-84f5-f2d1ae8f5404 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:14.493362 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb9875-573c-4364-84f5-f2d1ae8f5404 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:14.493629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb9875-573c-4364-84f5-f2d1ae8f5404 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:14.494334 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebb9875-573c-4364-84f5-f2d1ae8f5404 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:14.495989 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebb9875-573c-4364-84f5-f2d1ae8f5404 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:14.509348 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1134/2250] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:53:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:53:14.513457 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a901e0-9f87-4216-9587-341c7379c258 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:14.524274 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a901e0-9f87-4216-9587-341c7379c258 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:14.529112 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e95f70-552e-4570-8bd2-457d8daea647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:14.529798 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e95f70-552e-4570-8bd2-457d8daea647 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:14.530089 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e95f70-552e-4570-8bd2-457d8daea647 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:14.530390 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e95f70-552e-4570-8bd2-457d8daea647 admin admin] PATH_INFO: `/v3/projects/d695bcf5c67a409ba2a1845dd2163850/users/1c49a86c570d428ca9ba6eec86ab38c5/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:14.545853 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e95f70-552e-4570-8bd2-457d8daea647 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d695bcf5c67a409ba2a1845dd2163850, user_id=1c49a86c570d428ca9ba6eec86ab38c5, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:14.548553 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e95f70-552e-4570-8bd2-457d8daea647 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:14.561142 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1117/2251] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:53:14 2022] PUT /identity/v3/projects/d695bcf5c67a409ba2a1845dd2163850/users/1c49a86c570d428ca9ba6eec86ab38c5/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:53:14.571766 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af7514-22f6-4294-a4d1-eafe6c4769f6 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:14.572057 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af7514-22f6-4294-a4d1-eafe6c4769f6 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:14.572333 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af7514-22f6-4294-a4d1-eafe6c4769f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:14.605574 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-d7af7514-22f6-4294-a4d1-eafe6c4769f6 None None] MFA Rules not processed for user `1c49a86c570d428ca9ba6eec86ab38c5`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:14.606864 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-d7af7514-22f6-4294-a4d1-eafe6c4769f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:14.683594 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1135/2252] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:53:14.698870 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e95f70-552e-4570-8bd2-457d8daea647 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:14.700703 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-51e95f70-552e-4570-8bd2-457d8daea647 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:14.768485 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e95f70-552e-4570-8bd2-457d8daea647 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:14.773078 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec1ed45-4d85-458f-ba19-03e8e7f966e7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2b5853838dc40db89c46038c0606107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:14.773779 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1ed45-4d85-458f-ba19-03e8e7f966e7 service glance] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:14.774131 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1ed45-4d85-458f-ba19-03e8e7f966e7 service glance] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:14.774480 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1ed45-4d85-458f-ba19-03e8e7f966e7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:14.784583 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec1ed45-4d85-458f-ba19-03e8e7f966e7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:14.786165 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec1ed45-4d85-458f-ba19-03e8e7f966e7 service glance] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:14.796736 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1118/2253] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:53:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:53:15.045579 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7af7514-22f6-4294-a4d1-eafe6c4769f6 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:15.047102 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-d7af7514-22f6-4294-a4d1-eafe6c4769f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:15.112312 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:53:15.144429 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7af7514-22f6-4294-a4d1-eafe6c4769f6 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:15.156169 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39500018-7385-45a9-8636-28cc2b25fca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:15.156835 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39500018-7385-45a9-8636-28cc2b25fca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:15.157118 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39500018-7385-45a9-8636-28cc2b25fca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:15.157392 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39500018-7385-45a9-8636-28cc2b25fca9 admin admin] PATH_INFO: `/v3/users/35c01178fd4d416387b30e1067bf430d` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:15.163697 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39500018-7385-45a9-8636-28cc2b25fca9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35c01178fd4d416387b30e1067bf430d)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:15.164044 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39500018-7385-45a9-8636-28cc2b25fca9 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:15.240341 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-39500018-7385-45a9-8636-28cc2b25fca9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35c01178fd4d416387b30e1067bf430d'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:15.249080 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-39500018-7385-45a9-8636-28cc2b25fca9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35c01178fd4d416387b30e1067bf430d'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:15.250100 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-39500018-7385-45a9-8636-28cc2b25fca9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35c01178fd4d416387b30e1067bf430d'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:15.263681 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-39500018-7385-45a9-8636-28cc2b25fca9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35c01178fd4d416387b30e1067bf430d'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:15.296328 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-39500018-7385-45a9-8636-28cc2b25fca9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35c01178fd4d416387b30e1067bf430d'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:15.308732 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1136/2254] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:53:15 2022] DELETE /identity/v3/users/35c01178fd4d416387b30e1067bf430d => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:53:15.310073 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f3c59-0a3f-4ce4-9edb-38591134f35c None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:15.310547 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f3c59-0a3f-4ce4-9edb-38591134f35c None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:15.310809 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f3c59-0a3f-4ce4-9edb-38591134f35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:15.334509 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-881f3c59-0a3f-4ce4-9edb-38591134f35c None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:15.335940 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-881f3c59-0a3f-4ce4-9edb-38591134f35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:15.368751 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:53:15.414873 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1119/2255] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:53:15.424830 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39500018-7385-45a9-8636-28cc2b25fca9 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:15.427043 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-39500018-7385-45a9-8636-28cc2b25fca9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:15.485243 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39500018-7385-45a9-8636-28cc2b25fca9 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:15.489542 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9003ac-1f3a-4872-9c2d-6dbee75f7762 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:15.490161 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9003ac-1f3a-4872-9c2d-6dbee75f7762 service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:15.490444 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9003ac-1f3a-4872-9c2d-6dbee75f7762 service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:15.490718 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9003ac-1f3a-4872-9c2d-6dbee75f7762 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:15.500039 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9003ac-1f3a-4872-9c2d-6dbee75f7762 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:15.501534 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9003ac-1f3a-4872-9c2d-6dbee75f7762 service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:15.515307 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1137/2256] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:53:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 12:53:15.793566 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881f3c59-0a3f-4ce4-9edb-38591134f35c None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:15.795023 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-881f3c59-0a3f-4ce4-9edb-38591134f35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:15.808761 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881f3c59-0a3f-4ce4-9edb-38591134f35c None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:15.813711 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2389bc52-195b-4bb2-a145-d3c6874f7cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:15.814453 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389bc52-195b-4bb2-a145-d3c6874f7cce admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:15.814737 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389bc52-195b-4bb2-a145-d3c6874f7cce admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:15.815019 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389bc52-195b-4bb2-a145-d3c6874f7cce admin admin] PATH_INFO: `/v3/projects/5138c47bd23f488f903788264ea9ff58` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:15.816420 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2389bc52-195b-4bb2-a145-d3c6874f7cce admin admin] RBAC: Authorizing `identity:delete_project(project_id=5138c47bd23f488f903788264ea9ff58)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:15.818901 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2389bc52-195b-4bb2-a145-d3c6874f7cce admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:15.876795 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-2389bc52-195b-4bb2-a145-d3c6874f7cce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5138c47bd23f488f903788264ea9ff58', 'request_id': 'req-2389bc52-195b-4bb2-a145-d3c6874f7cce'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:15.883543 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-2389bc52-195b-4bb2-a145-d3c6874f7cce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5138c47bd23f488f903788264ea9ff58', 'request_id': 'req-2389bc52-195b-4bb2-a145-d3c6874f7cce'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:15.892518 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-2389bc52-195b-4bb2-a145-d3c6874f7cce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5138c47bd23f488f903788264ea9ff58', 'request_id': 'req-2389bc52-195b-4bb2-a145-d3c6874f7cce'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:15.913678 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-2389bc52-195b-4bb2-a145-d3c6874f7cce admin admin] The token cache is being invalidate because project 5138c47bd23f488f903788264ea9ff58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:53:15.913678 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-2389bc52-195b-4bb2-a145-d3c6874f7cce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:15.913678 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-2389bc52-195b-4bb2-a145-d3c6874f7cce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:15.914769 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1120/2257] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:53:15 2022] DELETE /identity/v3/projects/5138c47bd23f488f903788264ea9ff58 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:53:15.919712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc45dde-14e7-4e30-971c-97368784f86d None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:15.920004 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc45dde-14e7-4e30-971c-97368784f86d None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:15.920280 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc45dde-14e7-4e30-971c-97368784f86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:15.941289 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-6bc45dde-14e7-4e30-971c-97368784f86d None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:15.942542 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-6bc45dde-14e7-4e30-971c-97368784f86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:16.009104 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1138/2258] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:53:16.018711 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f610d-66a5-43b4-8004-fea2b108f468 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:16.019278 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f610d-66a5-43b4-8004-fea2b108f468 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:16.019706 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f610d-66a5-43b4-8004-fea2b108f468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:16.048649 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-379f610d-66a5-43b4-8004-fea2b108f468 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:16.049972 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-379f610d-66a5-43b4-8004-fea2b108f468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:16.058539 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1121/2259] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:53:16.062824 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc45dde-14e7-4e30-971c-97368784f86d None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:16.077799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc45dde-14e7-4e30-971c-97368784f86d None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:16.083320 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b794ce-7a2b-4d9b-90fe-0a2dec6cef24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:16.083990 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b794ce-7a2b-4d9b-90fe-0a2dec6cef24 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:16.084265 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b794ce-7a2b-4d9b-90fe-0a2dec6cef24 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:16.084535 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b794ce-7a2b-4d9b-90fe-0a2dec6cef24 admin admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:16.085237 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b794ce-7a2b-4d9b-90fe-0a2dec6cef24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:53:16.085539 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b794ce-7a2b-4d9b-90fe-0a2dec6cef24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:16.087140 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b794ce-7a2b-4d9b-90fe-0a2dec6cef24 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:16.095202 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:53:16.105596 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1139/2260] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:53:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:53:16.107176 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f610d-66a5-43b4-8004-fea2b108f468 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:16.117216 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f610d-66a5-43b4-8004-fea2b108f468 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:16.122036 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5adc7e4-0f89-44b6-a41f-666f3e701daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:16.122644 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5adc7e4-0f89-44b6-a41f-666f3e701daa admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:16.122925 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5adc7e4-0f89-44b6-a41f-666f3e701daa admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:16.123195 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5adc7e4-0f89-44b6-a41f-666f3e701daa admin admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:16.123989 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5adc7e4-0f89-44b6-a41f-666f3e701daa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:16.126156 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5adc7e4-0f89-44b6-a41f-666f3e701daa admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:16.154269 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1122/2261] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:53:16 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:53:16.158718 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b794ce-7a2b-4d9b-90fe-0a2dec6cef24 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:16.169833 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b794ce-7a2b-4d9b-90fe-0a2dec6cef24 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:16.235257 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e99f08-cb11-4a6d-a999-62452c5f45d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:16.235910 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e99f08-cb11-4a6d-a999-62452c5f45d3 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:16.236198 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e99f08-cb11-4a6d-a999-62452c5f45d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:16.236477 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e99f08-cb11-4a6d-a999-62452c5f45d3 admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:16.237152 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e99f08-cb11-4a6d-a999-62452c5f45d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:16.239124 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e99f08-cb11-4a6d-a999-62452c5f45d3 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:16.261856 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1140/2262] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:53:16 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:53:16.265859 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5adc7e4-0f89-44b6-a41f-666f3e701daa admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:16.276991 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5adc7e4-0f89-44b6-a41f-666f3e701daa admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:16.281851 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e63d4e-585d-490c-97f6-3e07c540b96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:16.282454 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e63d4e-585d-490c-97f6-3e07c540b96a admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:16.282734 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e63d4e-585d-490c-97f6-3e07c540b96a admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:16.283009 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e63d4e-585d-490c-97f6-3e07c540b96a admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:16.283803 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e63d4e-585d-490c-97f6-3e07c540b96a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:16.285395 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e63d4e-585d-490c-97f6-3e07c540b96a admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:16.295416 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1123/2263] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:53:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:53:16.299808 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e99f08-cb11-4a6d-a999-62452c5f45d3 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:16.310399 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e99f08-cb11-4a6d-a999-62452c5f45d3 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:16.314923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506532b4-3463-4ce7-8610-510bba981c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:16.315521 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506532b4-3463-4ce7-8610-510bba981c55 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:16.315904 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506532b4-3463-4ce7-8610-510bba981c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:16.316180 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506532b4-3463-4ce7-8610-510bba981c55 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:16.317409 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506532b4-3463-4ce7-8610-510bba981c55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:16.319029 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506532b4-3463-4ce7-8610-510bba981c55 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:16.331162 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1141/2264] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:53:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:53:16.336380 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e63d4e-585d-490c-97f6-3e07c540b96a admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:16.346547 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e63d4e-585d-490c-97f6-3e07c540b96a admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:16.351430 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba798c0-49ac-4ccb-ae52-3ae88304966c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:16.352165 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba798c0-49ac-4ccb-ae52-3ae88304966c admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:16.352415 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba798c0-49ac-4ccb-ae52-3ae88304966c admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:16.352692 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba798c0-49ac-4ccb-ae52-3ae88304966c admin admin] PATH_INFO: `/v3/projects/fbda6b838e3b4a8f89a693e4c5174695/users/bc8e48fa3a38429b9925b519ca28c1e2/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:16.369147 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba798c0-49ac-4ccb-ae52-3ae88304966c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbda6b838e3b4a8f89a693e4c5174695, user_id=bc8e48fa3a38429b9925b519ca28c1e2, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:16.372023 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba798c0-49ac-4ccb-ae52-3ae88304966c admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:16.387860 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1124/2265] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:53:16 2022] PUT /identity/v3/projects/fbda6b838e3b4a8f89a693e4c5174695/users/bc8e48fa3a38429b9925b519ca28c1e2/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:53:16.401526 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5710b-62d7-4731-b667-b37a1ead8058 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:16.401798 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5710b-62d7-4731-b667-b37a1ead8058 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:16.402483 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5710b-62d7-4731-b667-b37a1ead8058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:16.443169 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-dfd5710b-62d7-4731-b667-b37a1ead8058 None None] MFA Rules not processed for user `bc8e48fa3a38429b9925b519ca28c1e2`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:16.444429 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-dfd5710b-62d7-4731-b667-b37a1ead8058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:16.458437 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba798c0-49ac-4ccb-ae52-3ae88304966c admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:16.460596 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-5ba798c0-49ac-4ccb-ae52-3ae88304966c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:16.522180 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:53:16.527667 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1142/2266] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:53:16.540423 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba798c0-49ac-4ccb-ae52-3ae88304966c admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:16.543329 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd5710b-62d7-4731-b667-b37a1ead8058 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:16.544755 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-dfd5710b-62d7-4731-b667-b37a1ead8058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:16.547904 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1996705-bee9-4c9d-9d53-1eb3b892abad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:16.548567 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1996705-bee9-4c9d-9d53-1eb3b892abad admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:16.548855 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1996705-bee9-4c9d-9d53-1eb3b892abad admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:16.549191 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1996705-bee9-4c9d-9d53-1eb3b892abad admin admin] PATH_INFO: `/v3/users/e1fcebd110924d12a6da707dbf57f277` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:16.550818 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1996705-bee9-4c9d-9d53-1eb3b892abad admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1fcebd110924d12a6da707dbf57f277)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:16.554512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1996705-bee9-4c9d-9d53-1eb3b892abad admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:16.616773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd5710b-62d7-4731-b667-b37a1ead8058 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:16.620772 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd0f50f-2431-4198-b7d3-5975277aac42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:16.621444 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0f50f-2431-4198-b7d3-5975277aac42 service cinder] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:16.621766 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0f50f-2431-4198-b7d3-5975277aac42 service cinder] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:16.622039 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0f50f-2431-4198-b7d3-5975277aac42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:16.636084 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd0f50f-2431-4198-b7d3-5975277aac42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:16.636877 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-a1996705-bee9-4c9d-9d53-1eb3b892abad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1fcebd110924d12a6da707dbf57f277'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:16.638093 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd0f50f-2431-4198-b7d3-5975277aac42 service cinder] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:16.640265 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-a1996705-bee9-4c9d-9d53-1eb3b892abad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1fcebd110924d12a6da707dbf57f277'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:16.641114 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-a1996705-bee9-4c9d-9d53-1eb3b892abad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1fcebd110924d12a6da707dbf57f277'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:16.654458 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-a1996705-bee9-4c9d-9d53-1eb3b892abad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1fcebd110924d12a6da707dbf57f277'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:16.659489 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1143/2267] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:53:16 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 12:53:16.664534 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-a1996705-bee9-4c9d-9d53-1eb3b892abad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1fcebd110924d12a6da707dbf57f277'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:16.685453 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1125/2268] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:53:16 2022] DELETE /identity/v3/users/e1fcebd110924d12a6da707dbf57f277 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:53:16.691052 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800c448-ecf8-4a40-acea-ca1330f77865 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:16.692080 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800c448-ecf8-4a40-acea-ca1330f77865 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:16.692080 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800c448-ecf8-4a40-acea-ca1330f77865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:16.721759 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-1800c448-ecf8-4a40-acea-ca1330f77865 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:16.722993 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-1800c448-ecf8-4a40-acea-ca1330f77865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:16.728396 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:53:16.818923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1144/2269] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:53:16.829167 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1996705-bee9-4c9d-9d53-1eb3b892abad admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:16.830654 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-a1996705-bee9-4c9d-9d53-1eb3b892abad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:16.913430 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1996705-bee9-4c9d-9d53-1eb3b892abad admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:16.919460 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6026cab2-d43a-4bbe-b0ee-7df1371dbd31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:16.920817 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6026cab2-d43a-4bbe-b0ee-7df1371dbd31 service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:16.921339 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6026cab2-d43a-4bbe-b0ee-7df1371dbd31 service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:16.921764 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6026cab2-d43a-4bbe-b0ee-7df1371dbd31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:16.936758 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6026cab2-d43a-4bbe-b0ee-7df1371dbd31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:16.938264 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6026cab2-d43a-4bbe-b0ee-7df1371dbd31 service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:16.950919 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1126/2270] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:53:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:53:17.301454 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1800c448-ecf8-4a40-acea-ca1330f77865 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:17.303059 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-1800c448-ecf8-4a40-acea-ca1330f77865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:17.325068 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1800c448-ecf8-4a40-acea-ca1330f77865 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:17.333617 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681866a4-267e-4476-9dea-e2ab1cabab80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:17.334269 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681866a4-267e-4476-9dea-e2ab1cabab80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:17.334560 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681866a4-267e-4476-9dea-e2ab1cabab80 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:17.334850 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681866a4-267e-4476-9dea-e2ab1cabab80 admin admin] PATH_INFO: `/v3/projects/4ab951d5e91f4bf7a23a710cdc244360` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:17.336010 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681866a4-267e-4476-9dea-e2ab1cabab80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ab951d5e91f4bf7a23a710cdc244360)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:17.338160 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681866a4-267e-4476-9dea-e2ab1cabab80 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:17.397624 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-681866a4-267e-4476-9dea-e2ab1cabab80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ab951d5e91f4bf7a23a710cdc244360', 'request_id': 'req-681866a4-267e-4476-9dea-e2ab1cabab80'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:17.405399 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-681866a4-267e-4476-9dea-e2ab1cabab80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ab951d5e91f4bf7a23a710cdc244360', 'request_id': 'req-681866a4-267e-4476-9dea-e2ab1cabab80'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:17.414351 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-681866a4-267e-4476-9dea-e2ab1cabab80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ab951d5e91f4bf7a23a710cdc244360', 'request_id': 'req-681866a4-267e-4476-9dea-e2ab1cabab80'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:17.428466 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-681866a4-267e-4476-9dea-e2ab1cabab80 admin admin] The token cache is being invalidate because project 4ab951d5e91f4bf7a23a710cdc244360 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:53:17.428757 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-681866a4-267e-4476-9dea-e2ab1cabab80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:17.429684 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-681866a4-267e-4476-9dea-e2ab1cabab80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:17.431203 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1145/2271] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:53:17 2022] DELETE /identity/v3/projects/4ab951d5e91f4bf7a23a710cdc244360 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:53:17.442289 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3619b88c-7130-42a9-82d5-d6f0ab0d21a3 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:17.443100 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3619b88c-7130-42a9-82d5-d6f0ab0d21a3 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:17.443446 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3619b88c-7130-42a9-82d5-d6f0ab0d21a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:17.464665 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-3619b88c-7130-42a9-82d5-d6f0ab0d21a3 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:17.465974 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-3619b88c-7130-42a9-82d5-d6f0ab0d21a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:17.534816 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1127/2272] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:53:17.547479 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e57a3-98d3-4a94-9bc7-29468dedf457 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:17.547832 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e57a3-98d3-4a94-9bc7-29468dedf457 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:17.548167 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e57a3-98d3-4a94-9bc7-29468dedf457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:17.569409 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-063e57a3-98d3-4a94-9bc7-29468dedf457 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:17.570660 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-063e57a3-98d3-4a94-9bc7-29468dedf457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:17.579235 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1146/2273] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:53:17.583616 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3619b88c-7130-42a9-82d5-d6f0ab0d21a3 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:17.593406 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3619b88c-7130-42a9-82d5-d6f0ab0d21a3 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:17.597878 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79217d1b-89af-4d2c-973a-c5b05bc0692c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:17.599004 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79217d1b-89af-4d2c-973a-c5b05bc0692c admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:17.599455 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79217d1b-89af-4d2c-973a-c5b05bc0692c admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:17.599787 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79217d1b-89af-4d2c-973a-c5b05bc0692c admin admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:17.600508 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79217d1b-89af-4d2c-973a-c5b05bc0692c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:53:17.600834 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79217d1b-89af-4d2c-973a-c5b05bc0692c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:17.602548 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79217d1b-89af-4d2c-973a-c5b05bc0692c admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:17.611890 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:53:17.618694 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1128/2274] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:53:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:53:17.621429 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e57a3-98d3-4a94-9bc7-29468dedf457 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:17.631458 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e57a3-98d3-4a94-9bc7-29468dedf457 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:17.636145 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dca2a87-cb5e-4756-8314-0bb47da310fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:17.636752 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca2a87-cb5e-4756-8314-0bb47da310fe admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:17.637035 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca2a87-cb5e-4756-8314-0bb47da310fe admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:17.637309 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca2a87-cb5e-4756-8314-0bb47da310fe admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:17.637977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dca2a87-cb5e-4756-8314-0bb47da310fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:17.639917 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dca2a87-cb5e-4756-8314-0bb47da310fe admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:17.669295 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1147/2275] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:53:17 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:53:17.673604 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79217d1b-89af-4d2c-973a-c5b05bc0692c admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:17.684591 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79217d1b-89af-4d2c-973a-c5b05bc0692c admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:17.744835 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1747694d-44a9-4401-90f7-10801f2712ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:17.745568 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1747694d-44a9-4401-90f7-10801f2712ba admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:17.745845 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1747694d-44a9-4401-90f7-10801f2712ba admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:17.746123 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1747694d-44a9-4401-90f7-10801f2712ba admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:17.746879 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1747694d-44a9-4401-90f7-10801f2712ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:17.750709 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1747694d-44a9-4401-90f7-10801f2712ba admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:17.778111 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1129/2276] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:53:17 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:53:17.782775 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dca2a87-cb5e-4756-8314-0bb47da310fe admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:17.795994 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dca2a87-cb5e-4756-8314-0bb47da310fe admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:17.801821 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af2a58-3a1e-4683-b1c7-019c0ae9ec6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:17.802501 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af2a58-3a1e-4683-b1c7-019c0ae9ec6f admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:17.802781 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af2a58-3a1e-4683-b1c7-019c0ae9ec6f admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:17.803054 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af2a58-3a1e-4683-b1c7-019c0ae9ec6f admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:17.803791 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af2a58-3a1e-4683-b1c7-019c0ae9ec6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:17.805410 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af2a58-3a1e-4683-b1c7-019c0ae9ec6f admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:17.816111 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1148/2277] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:53:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:53:17.820183 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1747694d-44a9-4401-90f7-10801f2712ba admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:17.830350 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1747694d-44a9-4401-90f7-10801f2712ba admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:17.835938 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0938ef4-a677-4b78-9c71-b03329c7f54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:17.836615 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0938ef4-a677-4b78-9c71-b03329c7f54a admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:17.836959 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0938ef4-a677-4b78-9c71-b03329c7f54a admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:17.837288 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0938ef4-a677-4b78-9c71-b03329c7f54a admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:17.838428 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0938ef4-a677-4b78-9c71-b03329c7f54a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:17.840298 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0938ef4-a677-4b78-9c71-b03329c7f54a admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:17.851856 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1130/2278] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:53:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:53:17.856128 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af2a58-3a1e-4683-b1c7-019c0ae9ec6f admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:17.867112 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af2a58-3a1e-4683-b1c7-019c0ae9ec6f admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:17.871811 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54381a9a-95a2-40ad-848e-79cec14b7fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:17.872484 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54381a9a-95a2-40ad-848e-79cec14b7fae admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:17.872768 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54381a9a-95a2-40ad-848e-79cec14b7fae admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:17.873046 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54381a9a-95a2-40ad-848e-79cec14b7fae admin admin] PATH_INFO: `/v3/projects/fa4a3dadafc44483ad16923db53db457/users/7a6832c70c0c401985bc4f26a787080f/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:17.888246 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54381a9a-95a2-40ad-848e-79cec14b7fae admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa4a3dadafc44483ad16923db53db457, user_id=7a6832c70c0c401985bc4f26a787080f, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:17.891013 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54381a9a-95a2-40ad-848e-79cec14b7fae admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:17.906108 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1149/2279] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:53:17 2022] PUT /identity/v3/projects/fa4a3dadafc44483ad16923db53db457/users/7a6832c70c0c401985bc4f26a787080f/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:53:17.911889 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a072c6d-3a68-441f-b2de-212b9cf49d3d None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:17.912211 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a072c6d-3a68-441f-b2de-212b9cf49d3d None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:17.912488 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a072c6d-3a68-441f-b2de-212b9cf49d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:17.933123 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-4a072c6d-3a68-441f-b2de-212b9cf49d3d None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:17.934370 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-4a072c6d-3a68-441f-b2de-212b9cf49d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:17.994920 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1131/2280] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:53:18.005382 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54381a9a-95a2-40ad-848e-79cec14b7fae admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:18.007290 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-54381a9a-95a2-40ad-848e-79cec14b7fae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:18.045189 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:53:18.075345 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54381a9a-95a2-40ad-848e-79cec14b7fae admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:18.079018 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d711f8e-0f76-4196-9c09-3ad78123daeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:18.079653 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d711f8e-0f76-4196-9c09-3ad78123daeb service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:18.079946 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d711f8e-0f76-4196-9c09-3ad78123daeb service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:18.080215 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d711f8e-0f76-4196-9c09-3ad78123daeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:18.090448 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d711f8e-0f76-4196-9c09-3ad78123daeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:18.091905 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d711f8e-0f76-4196-9c09-3ad78123daeb service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:18.104775 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1150/2281] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:53:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:53:18.159894 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7e964-8c40-45a9-822d-d1f5ae87886f None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:18.160448 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7e964-8c40-45a9-822d-d1f5ae87886f None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:18.161479 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7e964-8c40-45a9-822d-d1f5ae87886f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:18.183106 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-6ac7e964-8c40-45a9-822d-d1f5ae87886f None None] MFA Rules not processed for user `bc8e48fa3a38429b9925b519ca28c1e2`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:18.184345 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-6ac7e964-8c40-45a9-822d-d1f5ae87886f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:18.256059 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1132/2282] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:53:18.904986 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f3570-ad98-4a73-a544-ebc23b43dbac None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:18.905530 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f3570-ad98-4a73-a544-ebc23b43dbac None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:18.905530 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f3570-ad98-4a73-a544-ebc23b43dbac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:18.941801 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-e38f3570-ad98-4a73-a544-ebc23b43dbac None None] MFA Rules not processed for user `7a6832c70c0c401985bc4f26a787080f`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:18.944199 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-e38f3570-ad98-4a73-a544-ebc23b43dbac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:19.015179 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1151/2283] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:53:19.027570 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac7e964-8c40-45a9-822d-d1f5ae87886f None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:19.029033 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-6ac7e964-8c40-45a9-822d-d1f5ae87886f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:19.057634 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:53:19.091868 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac7e964-8c40-45a9-822d-d1f5ae87886f None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:19.095663 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb79f07-27de-4568-80eb-8687808b98a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:19.096341 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb79f07-27de-4568-80eb-8687808b98a2 service cinder] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:19.096626 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb79f07-27de-4568-80eb-8687808b98a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:19.096902 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb79f07-27de-4568-80eb-8687808b98a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:19.106390 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb79f07-27de-4568-80eb-8687808b98a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:19.107934 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb79f07-27de-4568-80eb-8687808b98a2 service cinder] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:19.118040 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1133/2284] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:53:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:53:20.496046 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2861771-1ce6-4b58-b899-8262d59dd3e2 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:20.496575 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2861771-1ce6-4b58-b899-8262d59dd3e2 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:20.496575 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2861771-1ce6-4b58-b899-8262d59dd3e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:20.516423 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-a2861771-1ce6-4b58-b899-8262d59dd3e2 None None] MFA Rules not processed for user `7a6832c70c0c401985bc4f26a787080f`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:20.517664 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-a2861771-1ce6-4b58-b899-8262d59dd3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:20.524598 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1152/2285] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:53:29.113367 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=15 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:53:30.512545 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=48 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:53:38.749168 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3aaf1-349a-4f6f-9d67-bec63e2e64b7 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:38.749935 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3aaf1-349a-4f6f-9d67-bec63e2e64b7 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:38.750030 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3aaf1-349a-4f6f-9d67-bec63e2e64b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:38.770198 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-9bc3aaf1-349a-4f6f-9d67-bec63e2e64b7 None None] MFA Rules not processed for user `2b5894762d4b4cc99521bc224d2bddd7`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:38.771363 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-9bc3aaf1-349a-4f6f-9d67-bec63e2e64b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:38.852475 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1134/2286] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:53:38.985379 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f4fd8-b00a-44b9-92a7-55f46f62c19f None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:38.986247 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f4fd8-b00a-44b9-92a7-55f46f62c19f None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:38.986529 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f4fd8-b00a-44b9-92a7-55f46f62c19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:39.005967 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-a53f4fd8-b00a-44b9-92a7-55f46f62c19f None None] MFA Rules not processed for user `1401efb776ab424e8ff4f09eac33d92f`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:39.007385 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-a53f4fd8-b00a-44b9-92a7-55f46f62c19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:39.091076 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1153/2287] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:53:48.843968 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=25 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:53:49.077657 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=25 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:53:51.493414 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3aaf1-349a-4f6f-9d67-bec63e2e64b7 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:51.494964 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-9bc3aaf1-349a-4f6f-9d67-bec63e2e64b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:51.511184 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3aaf1-349a-4f6f-9d67-bec63e2e64b7 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:51.516666 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac007ebe-5b56-4b03-ab4c-d0fff9d17e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:51.517919 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac007ebe-5b56-4b03-ab4c-d0fff9d17e75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:51.518393 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac007ebe-5b56-4b03-ab4c-d0fff9d17e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:51.518854 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac007ebe-5b56-4b03-ab4c-d0fff9d17e75 admin admin] PATH_INFO: `/v3/users/7fe15bf80eed482abe1338f107a1992f` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:51.520388 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac007ebe-5b56-4b03-ab4c-d0fff9d17e75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fe15bf80eed482abe1338f107a1992f)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:51.524042 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac007ebe-5b56-4b03-ab4c-d0fff9d17e75 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:51.633146 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-ac007ebe-5b56-4b03-ab4c-d0fff9d17e75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fe15bf80eed482abe1338f107a1992f'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:51.634126 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-ac007ebe-5b56-4b03-ab4c-d0fff9d17e75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fe15bf80eed482abe1338f107a1992f'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:51.634946 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-ac007ebe-5b56-4b03-ab4c-d0fff9d17e75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fe15bf80eed482abe1338f107a1992f'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:51.659631 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-ac007ebe-5b56-4b03-ab4c-d0fff9d17e75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fe15bf80eed482abe1338f107a1992f'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:51.668150 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-ac007ebe-5b56-4b03-ab4c-d0fff9d17e75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fe15bf80eed482abe1338f107a1992f'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:51.684813 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1135/2288] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:53:51 2022] DELETE /identity/v3/users/7fe15bf80eed482abe1338f107a1992f => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:53:51.689496 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8accdad0-34d4-466f-a6e6-e606a3c18205 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:51.690353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8accdad0-34d4-466f-a6e6-e606a3c18205 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:51.690814 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8accdad0-34d4-466f-a6e6-e606a3c18205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:51.724822 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-8accdad0-34d4-466f-a6e6-e606a3c18205 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:51.726569 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-8accdad0-34d4-466f-a6e6-e606a3c18205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:51.812741 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1154/2289] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:53:51.823731 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac007ebe-5b56-4b03-ab4c-d0fff9d17e75 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:51.833655 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-ac007ebe-5b56-4b03-ab4c-d0fff9d17e75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:51.912989 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac007ebe-5b56-4b03-ab4c-d0fff9d17e75 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:51.918058 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4ca227-fdbf-48eb-9b80-cadd38ad4ff7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:51.918800 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ca227-fdbf-48eb-9b80-cadd38ad4ff7 service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:51.919346 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ca227-fdbf-48eb-9b80-cadd38ad4ff7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:51.919911 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ca227-fdbf-48eb-9b80-cadd38ad4ff7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:51.932391 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4ca227-fdbf-48eb-9b80-cadd38ad4ff7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:51.935210 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4ca227-fdbf-48eb-9b80-cadd38ad4ff7 service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:51.948122 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1136/2290] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:53:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:53:52.295529 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8accdad0-34d4-466f-a6e6-e606a3c18205 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:52.297124 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-8accdad0-34d4-466f-a6e6-e606a3c18205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:52.310689 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8accdad0-34d4-466f-a6e6-e606a3c18205 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:52.316159 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989816e8-37c3-4c3d-86f5-d8659b8a94ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:52.317069 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989816e8-37c3-4c3d-86f5-d8659b8a94ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:52.318038 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989816e8-37c3-4c3d-86f5-d8659b8a94ea admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:52.318244 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989816e8-37c3-4c3d-86f5-d8659b8a94ea admin admin] PATH_INFO: `/v3/projects/e80027d473534e0ab00fa91b53221cb3` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:52.319900 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989816e8-37c3-4c3d-86f5-d8659b8a94ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=e80027d473534e0ab00fa91b53221cb3)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:52.321847 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989816e8-37c3-4c3d-86f5-d8659b8a94ea admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:52.401935 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-989816e8-37c3-4c3d-86f5-d8659b8a94ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e80027d473534e0ab00fa91b53221cb3', 'request_id': 'req-989816e8-37c3-4c3d-86f5-d8659b8a94ea'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:52.427808 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-989816e8-37c3-4c3d-86f5-d8659b8a94ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e80027d473534e0ab00fa91b53221cb3', 'request_id': 'req-989816e8-37c3-4c3d-86f5-d8659b8a94ea'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:52.437504 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-989816e8-37c3-4c3d-86f5-d8659b8a94ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e80027d473534e0ab00fa91b53221cb3', 'request_id': 'req-989816e8-37c3-4c3d-86f5-d8659b8a94ea'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:52.463560 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-989816e8-37c3-4c3d-86f5-d8659b8a94ea admin admin] The token cache is being invalidate because project e80027d473534e0ab00fa91b53221cb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:53:52.464130 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-989816e8-37c3-4c3d-86f5-d8659b8a94ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:52.465075 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-989816e8-37c3-4c3d-86f5-d8659b8a94ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:52.466995 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1155/2291] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:53:52 2022] DELETE /identity/v3/projects/e80027d473534e0ab00fa91b53221cb3 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:53:52.471854 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad528656-3838-4574-a9c3-52cce8691231 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:52.472527 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad528656-3838-4574-a9c3-52cce8691231 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:52.472899 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad528656-3838-4574-a9c3-52cce8691231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:52.510581 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-ad528656-3838-4574-a9c3-52cce8691231 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:52.512569 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-ad528656-3838-4574-a9c3-52cce8691231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:52.644264 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1137/2292] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:53:52.645876 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe341b7a-8e11-4b0e-8779-221eb6739077 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:52.646190 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe341b7a-8e11-4b0e-8779-221eb6739077 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:52.646585 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe341b7a-8e11-4b0e-8779-221eb6739077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:52.674040 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-fe341b7a-8e11-4b0e-8779-221eb6739077 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:52.675400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-fe341b7a-8e11-4b0e-8779-221eb6739077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:52.690719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1156/2293] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:53:52.695427 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad528656-3838-4574-a9c3-52cce8691231 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:52.712019 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad528656-3838-4574-a9c3-52cce8691231 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:52.736166 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc8486-46c3-49ad-949b-92f75c821d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:52.737136 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc8486-46c3-49ad-949b-92f75c821d9c admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:52.737543 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc8486-46c3-49ad-949b-92f75c821d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:52.738073 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc8486-46c3-49ad-949b-92f75c821d9c admin admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:52.739514 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fc8486-46c3-49ad-949b-92f75c821d9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:53:52.739604 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fc8486-46c3-49ad-949b-92f75c821d9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:52.744861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fc8486-46c3-49ad-949b-92f75c821d9c admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:52.772373 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1138/2294] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:53:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:53:52.774851 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe341b7a-8e11-4b0e-8779-221eb6739077 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:52.793138 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe341b7a-8e11-4b0e-8779-221eb6739077 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:52.800172 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4325b4-1698-44d7-8726-3604b85c4222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:52.801831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4325b4-1698-44d7-8726-3604b85c4222 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:52.802381 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4325b4-1698-44d7-8726-3604b85c4222 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:52.803594 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4325b4-1698-44d7-8726-3604b85c4222 admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:52.805403 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4325b4-1698-44d7-8726-3604b85c4222 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:52.808589 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4325b4-1698-44d7-8726-3604b85c4222 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:52.873086 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1157/2295] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:53:52 2022] POST /identity/v3/projects => generated 375 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:53:52.874201 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc8486-46c3-49ad-949b-92f75c821d9c admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:52.880681 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:53:52.904172 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc8486-46c3-49ad-949b-92f75c821d9c admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:52.911058 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65786c63-39a9-4be3-ad6c-93573ec1aecb None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:52.911415 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65786c63-39a9-4be3-ad6c-93573ec1aecb None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:52.911726 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65786c63-39a9-4be3-ad6c-93573ec1aecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:52.950820 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-65786c63-39a9-4be3-ad6c-93573ec1aecb None None] MFA Rules not processed for user `30fc532f464843a3ad35e4d3c387bb3a`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:52.953454 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-65786c63-39a9-4be3-ad6c-93573ec1aecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:53.041098 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932d70a-702f-434b-8d8b-2649b776f8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:53.041942 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932d70a-702f-434b-8d8b-2649b776f8c0 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:53.042305 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932d70a-702f-434b-8d8b-2649b776f8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:53.043868 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932d70a-702f-434b-8d8b-2649b776f8c0 admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:53.044763 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3932d70a-702f-434b-8d8b-2649b776f8c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:53.047205 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3932d70a-702f-434b-8d8b-2649b776f8c0 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:53.050201 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:53:53.075499 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1158/2296] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Jul 26 12:53:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:53:53.095989 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1139/2297] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:53:52 2022] POST /identity/v3/users => generated 288 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:53:53.097183 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65786c63-39a9-4be3-ad6c-93573ec1aecb None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:53.114060 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65786c63-39a9-4be3-ad6c-93573ec1aecb None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:53.147561 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c1ffe3-1dfd-40bb-a2a8-bb2ca858d76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:53.150182 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932d70a-702f-434b-8d8b-2649b776f8c0 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:53.150415 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c1ffe3-1dfd-40bb-a2a8-bb2ca858d76e admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:53.150692 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c1ffe3-1dfd-40bb-a2a8-bb2ca858d76e admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:53.150974 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c1ffe3-1dfd-40bb-a2a8-bb2ca858d76e admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:53.151869 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c1ffe3-1dfd-40bb-a2a8-bb2ca858d76e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:53.152336 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-3932d70a-702f-434b-8d8b-2649b776f8c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:53.153523 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c1ffe3-1dfd-40bb-a2a8-bb2ca858d76e admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:53.186659 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1159/2298] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:53:53 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:53:53.190875 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c1ffe3-1dfd-40bb-a2a8-bb2ca858d76e admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:53.207414 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c1ffe3-1dfd-40bb-a2a8-bb2ca858d76e admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:53.219985 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a220314-c1cb-4234-ac3f-cea74df8b3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:53.221035 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a220314-c1cb-4234-ac3f-cea74df8b3d7 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:53.221843 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a220314-c1cb-4234-ac3f-cea74df8b3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:53.222323 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a220314-c1cb-4234-ac3f-cea74df8b3d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:53.223216 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a220314-c1cb-4234-ac3f-cea74df8b3d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:53.226816 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a220314-c1cb-4234-ac3f-cea74df8b3d7 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:53.257957 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1160/2299] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:53:53 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 12:53:53.264286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a220314-c1cb-4234-ac3f-cea74df8b3d7 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:53.293802 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a220314-c1cb-4234-ac3f-cea74df8b3d7 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:53.314053 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932d70a-702f-434b-8d8b-2649b776f8c0 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:53.320630 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce3405b-48b8-4dfb-99b9-f014011755fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:53.321793 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce3405b-48b8-4dfb-99b9-f014011755fc admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:53.322951 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba40c846-2f3a-443f-9a26-b586e4f1ebc7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd2483f74d7d48b1a7b8ffe32f63abde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:53.323729 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40c846-2f3a-443f-9a26-b586e4f1ebc7 service swift] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:53.324037 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce3405b-48b8-4dfb-99b9-f014011755fc admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:53.324489 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce3405b-48b8-4dfb-99b9-f014011755fc admin admin] PATH_INFO: `/v3/projects/ca9be6b3f09745ac84a69ea4d4b9cc79/users/b8c3094e6799451dbe310dc5ddbe0155/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:53.324843 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40c846-2f3a-443f-9a26-b586e4f1ebc7 service swift] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:53.325226 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40c846-2f3a-443f-9a26-b586e4f1ebc7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:53.338058 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba40c846-2f3a-443f-9a26-b586e4f1ebc7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:53.340369 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba40c846-2f3a-443f-9a26-b586e4f1ebc7 service swift] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:53.352336 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1140/2300] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Jul 26 12:53:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 12:53:53.368091 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce3405b-48b8-4dfb-99b9-f014011755fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca9be6b3f09745ac84a69ea4d4b9cc79, user_id=b8c3094e6799451dbe310dc5ddbe0155, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:53.372008 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce3405b-48b8-4dfb-99b9-f014011755fc admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:53.396235 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1161/2301] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:53:53 2022] PUT /identity/v3/projects/ca9be6b3f09745ac84a69ea4d4b9cc79/users/b8c3094e6799451dbe310dc5ddbe0155/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:53:53.402635 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0469b-3185-406a-8dd8-83f3c77e1f06 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:53.402924 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0469b-3185-406a-8dd8-83f3c77e1f06 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:53.403202 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0469b-3185-406a-8dd8-83f3c77e1f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:53.465006 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-92d0469b-3185-406a-8dd8-83f3c77e1f06 None None] MFA Rules not processed for user `b8c3094e6799451dbe310dc5ddbe0155`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:53.467623 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-92d0469b-3185-406a-8dd8-83f3c77e1f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:53.634534 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce3405b-48b8-4dfb-99b9-f014011755fc admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:53.635735 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1141/2302] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:53:53.636684 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-2ce3405b-48b8-4dfb-99b9-f014011755fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:53.736565 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce3405b-48b8-4dfb-99b9-f014011755fc admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:53.744426 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2fa8b1-6885-41b1-9208-6799ea85f412 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:53.746777 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fa8b1-6885-41b1-9208-6799ea85f412 service cinder] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:53.747223 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fa8b1-6885-41b1-9208-6799ea85f412 service cinder] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:53.748221 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fa8b1-6885-41b1-9208-6799ea85f412 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:53.759301 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2fa8b1-6885-41b1-9208-6799ea85f412 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:53.761603 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2fa8b1-6885-41b1-9208-6799ea85f412 service cinder] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:53.779243 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1162/2303] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:53:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:53:56.472795 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0469b-3185-406a-8dd8-83f3c77e1f06 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:56.475029 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-92d0469b-3185-406a-8dd8-83f3c77e1f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:56.504659 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:53:56.613225 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0469b-3185-406a-8dd8-83f3c77e1f06 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:56.618542 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a96f6-0ae7-4897-93b3-5e43b417e815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:56.619624 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a96f6-0ae7-4897-93b3-5e43b417e815 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:56.620142 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a96f6-0ae7-4897-93b3-5e43b417e815 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:56.620666 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a96f6-0ae7-4897-93b3-5e43b417e815 admin admin] PATH_INFO: `/v3/users/1c49a86c570d428ca9ba6eec86ab38c5` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:56.621790 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33a96f6-0ae7-4897-93b3-5e43b417e815 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c49a86c570d428ca9ba6eec86ab38c5)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:56.624865 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33a96f6-0ae7-4897-93b3-5e43b417e815 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:56.700270 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-b33a96f6-0ae7-4897-93b3-5e43b417e815 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c49a86c570d428ca9ba6eec86ab38c5'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:56.702225 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-b33a96f6-0ae7-4897-93b3-5e43b417e815 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c49a86c570d428ca9ba6eec86ab38c5'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:56.703422 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-b33a96f6-0ae7-4897-93b3-5e43b417e815 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c49a86c570d428ca9ba6eec86ab38c5'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:56.722077 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-b33a96f6-0ae7-4897-93b3-5e43b417e815 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c49a86c570d428ca9ba6eec86ab38c5'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:56.737194 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-b33a96f6-0ae7-4897-93b3-5e43b417e815 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c49a86c570d428ca9ba6eec86ab38c5'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:56.763687 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1142/2304] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:53:56 2022] DELETE /identity/v3/users/1c49a86c570d428ca9ba6eec86ab38c5 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:53:56.769094 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9684d9a-e6b4-4bde-b697-dbd8e3f657ed None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:56.769689 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9684d9a-e6b4-4bde-b697-dbd8e3f657ed None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:56.769689 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9684d9a-e6b4-4bde-b697-dbd8e3f657ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:56.818987 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-f9684d9a-e6b4-4bde-b697-dbd8e3f657ed None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:56.820595 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-f9684d9a-e6b4-4bde-b697-dbd8e3f657ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:56.915775 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1163/2305] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:53:56.923312 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a96f6-0ae7-4897-93b3-5e43b417e815 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:56.929508 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-b33a96f6-0ae7-4897-93b3-5e43b417e815 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:57.050500 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a96f6-0ae7-4897-93b3-5e43b417e815 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:57.055101 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50c579b-6f4e-4e05-bcfb-bb9bbdfe027f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:57.056101 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c579b-6f4e-4e05-bcfb-bb9bbdfe027f service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:57.056624 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c579b-6f4e-4e05-bcfb-bb9bbdfe027f service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:57.057367 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c579b-6f4e-4e05-bcfb-bb9bbdfe027f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:57.070901 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50c579b-6f4e-4e05-bcfb-bb9bbdfe027f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:57.072506 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50c579b-6f4e-4e05-bcfb-bb9bbdfe027f service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:57.084884 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1143/2306] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:53:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:53:57.710663 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9684d9a-e6b4-4bde-b697-dbd8e3f657ed None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:57.742547 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-f9684d9a-e6b4-4bde-b697-dbd8e3f657ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:57.856288 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9684d9a-e6b4-4bde-b697-dbd8e3f657ed None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:57.869901 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62223e25-f25b-49c1-9caf-f1c16d295a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:57.872565 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62223e25-f25b-49c1-9caf-f1c16d295a51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:57.873430 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62223e25-f25b-49c1-9caf-f1c16d295a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:57.874037 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62223e25-f25b-49c1-9caf-f1c16d295a51 admin admin] PATH_INFO: `/v3/projects/d695bcf5c67a409ba2a1845dd2163850` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:57.879259 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62223e25-f25b-49c1-9caf-f1c16d295a51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d695bcf5c67a409ba2a1845dd2163850)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:57.881911 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62223e25-f25b-49c1-9caf-f1c16d295a51 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:58.080201 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-62223e25-f25b-49c1-9caf-f1c16d295a51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd695bcf5c67a409ba2a1845dd2163850', 'request_id': 'req-62223e25-f25b-49c1-9caf-f1c16d295a51'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:58.087494 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:53:58.105470 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-62223e25-f25b-49c1-9caf-f1c16d295a51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd695bcf5c67a409ba2a1845dd2163850', 'request_id': 'req-62223e25-f25b-49c1-9caf-f1c16d295a51'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:58.120475 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-62223e25-f25b-49c1-9caf-f1c16d295a51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd695bcf5c67a409ba2a1845dd2163850', 'request_id': 'req-62223e25-f25b-49c1-9caf-f1c16d295a51'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:58.175067 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-62223e25-f25b-49c1-9caf-f1c16d295a51 admin admin] The token cache is being invalidate because project d695bcf5c67a409ba2a1845dd2163850 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:53:58.175353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-62223e25-f25b-49c1-9caf-f1c16d295a51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:58.176233 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-62223e25-f25b-49c1-9caf-f1c16d295a51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:53:58.180147 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1164/2307] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:53:57 2022] DELETE /identity/v3/projects/d695bcf5c67a409ba2a1845dd2163850 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:53:58.185735 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed3dea-bea8-4050-b50c-06f31b59e3b6 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:58.186741 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed3dea-bea8-4050-b50c-06f31b59e3b6 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:58.187199 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed3dea-bea8-4050-b50c-06f31b59e3b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:58.250261 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-abed3dea-bea8-4050-b50c-06f31b59e3b6 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:58.252513 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-abed3dea-bea8-4050-b50c-06f31b59e3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:58.294143 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:53:58.341156 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1144/2308] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:53:58.350263 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9373ed-7260-49ef-8dca-c4536c9a9e6b None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:58.350542 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9373ed-7260-49ef-8dca-c4536c9a9e6b None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:58.350865 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9373ed-7260-49ef-8dca-c4536c9a9e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:58.374347 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-bf9373ed-7260-49ef-8dca-c4536c9a9e6b None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:58.376140 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-bf9373ed-7260-49ef-8dca-c4536c9a9e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:58.390357 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1165/2309] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:53:58.395099 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed3dea-bea8-4050-b50c-06f31b59e3b6 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:58.418286 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed3dea-bea8-4050-b50c-06f31b59e3b6 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:58.424579 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f730b94-fc29-4499-ba7a-db0543b9c3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:58.425201 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f730b94-fc29-4499-ba7a-db0543b9c3fc admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:58.425482 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f730b94-fc29-4499-ba7a-db0543b9c3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:58.426011 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f730b94-fc29-4499-ba7a-db0543b9c3fc admin admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:58.427253 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f730b94-fc29-4499-ba7a-db0543b9c3fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:53:58.427561 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f730b94-fc29-4499-ba7a-db0543b9c3fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:58.429265 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f730b94-fc29-4499-ba7a-db0543b9c3fc admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:58.446297 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1145/2310] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:53:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:53:58.450191 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9373ed-7260-49ef-8dca-c4536c9a9e6b None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:58.460749 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9373ed-7260-49ef-8dca-c4536c9a9e6b None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:58.471236 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c97271e-233b-4239-b294-8fb3466f6d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:58.472039 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97271e-233b-4239-b294-8fb3466f6d74 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:58.472327 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97271e-233b-4239-b294-8fb3466f6d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:58.472789 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97271e-233b-4239-b294-8fb3466f6d74 admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:58.474112 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c97271e-233b-4239-b294-8fb3466f6d74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:58.476857 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c97271e-233b-4239-b294-8fb3466f6d74 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:58.513891 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1166/2311] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:53:58 2022] POST /identity/v3/projects => generated 399 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:53:58.515203 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f730b94-fc29-4499-ba7a-db0543b9c3fc admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:58.528763 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f730b94-fc29-4499-ba7a-db0543b9c3fc admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:58.601210 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7227c5c-bbba-495b-aebd-520d63de6b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:58.602137 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7227c5c-bbba-495b-aebd-520d63de6b40 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:58.602664 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7227c5c-bbba-495b-aebd-520d63de6b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:58.603131 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7227c5c-bbba-495b-aebd-520d63de6b40 admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:58.604018 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7227c5c-bbba-495b-aebd-520d63de6b40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:58.606089 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7227c5c-bbba-495b-aebd-520d63de6b40 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:58.635681 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1146/2312] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:53:58 2022] POST /identity/v3/users => generated 300 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:53:58.638073 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c97271e-233b-4239-b294-8fb3466f6d74 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:58.650369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c97271e-233b-4239-b294-8fb3466f6d74 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:58.655408 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aeffc4-e8e1-4f3c-88cf-b51f41660e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:58.656160 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aeffc4-e8e1-4f3c-88cf-b51f41660e6d admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:58.656637 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aeffc4-e8e1-4f3c-88cf-b51f41660e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:58.657089 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aeffc4-e8e1-4f3c-88cf-b51f41660e6d admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:58.657795 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aeffc4-e8e1-4f3c-88cf-b51f41660e6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:58.660318 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aeffc4-e8e1-4f3c-88cf-b51f41660e6d admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:58.677918 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1167/2313] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:53:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 12:53:58.682142 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7227c5c-bbba-495b-aebd-520d63de6b40 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:58.693454 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7227c5c-bbba-495b-aebd-520d63de6b40 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:58.698149 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe4ca0-fc90-4ddf-8756-64649f6e73f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:58.698754 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe4ca0-fc90-4ddf-8756-64649f6e73f2 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:58.699043 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe4ca0-fc90-4ddf-8756-64649f6e73f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:58.699320 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe4ca0-fc90-4ddf-8756-64649f6e73f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:58.700092 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfe4ca0-fc90-4ddf-8756-64649f6e73f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:58.701912 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfe4ca0-fc90-4ddf-8756-64649f6e73f2 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:58.718505 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1147/2314] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:53:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:53:58.720111 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aeffc4-e8e1-4f3c-88cf-b51f41660e6d admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:58.738445 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aeffc4-e8e1-4f3c-88cf-b51f41660e6d admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:58.744603 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86afc33f-3007-40f4-9030-cbf22c1169a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:58.745282 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86afc33f-3007-40f4-9030-cbf22c1169a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:58.745565 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86afc33f-3007-40f4-9030-cbf22c1169a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:58.745842 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86afc33f-3007-40f4-9030-cbf22c1169a9 admin admin] PATH_INFO: `/v3/projects/d04db2b45b39477296aca9dfad7ddfc4/users/fd8c9fec30ef47e3b3ca020027efb91c/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:58.764384 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86afc33f-3007-40f4-9030-cbf22c1169a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d04db2b45b39477296aca9dfad7ddfc4, user_id=fd8c9fec30ef47e3b3ca020027efb91c, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:58.767318 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86afc33f-3007-40f4-9030-cbf22c1169a9 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:58.784660 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1168/2315] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:53:58 2022] PUT /identity/v3/projects/d04db2b45b39477296aca9dfad7ddfc4/users/fd8c9fec30ef47e3b3ca020027efb91c/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:53:58.795898 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ac647-d409-4860-9fdc-f9bc9341a2e5 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:58.796481 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ac647-d409-4860-9fdc-f9bc9341a2e5 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:58.797032 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ac647-d409-4860-9fdc-f9bc9341a2e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:58.839601 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-414ac647-d409-4860-9fdc-f9bc9341a2e5 None None] MFA Rules not processed for user `fd8c9fec30ef47e3b3ca020027efb91c`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:53:58.840825 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-414ac647-d409-4860-9fdc-f9bc9341a2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:59.007540 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1148/2316] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:53:58 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:53:59.038832 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86afc33f-3007-40f4-9030-cbf22c1169a9 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:53:59.040885 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-86afc33f-3007-40f4-9030-cbf22c1169a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:53:59.138536 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86afc33f-3007-40f4-9030-cbf22c1169a9 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:53:59.142812 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfdc22a-bc60-4ce0-a97e-c8c48b0b330a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:53:59.146285 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfdc22a-bc60-4ce0-a97e-c8c48b0b330a service cinder] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:53:59.146285 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfdc22a-bc60-4ce0-a97e-c8c48b0b330a service cinder] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:53:59.146285 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfdc22a-bc60-4ce0-a97e-c8c48b0b330a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:53:59.178337 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfdc22a-bc60-4ce0-a97e-c8c48b0b330a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:53:59.181180 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfdc22a-bc60-4ce0-a97e-c8c48b0b330a service cinder] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:53:59.223297 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1169/2317] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:53:59 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:54:02.611463 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c564fa-af9c-440c-bdf9-d3cda9d49b04 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:02.612527 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c564fa-af9c-440c-bdf9-d3cda9d49b04 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:02.613150 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c564fa-af9c-440c-bdf9-d3cda9d49b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:02.686709 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-35c564fa-af9c-440c-bdf9-d3cda9d49b04 None None] MFA Rules not processed for user `30fc532f464843a3ad35e4d3c387bb3a`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:54:02.687996 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-35c564fa-af9c-440c-bdf9-d3cda9d49b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:02.766713 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:54:02.860830 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1149/2318] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Jul 26 12:54:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:54:02.873022 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfdc22a-bc60-4ce0-a97e-c8c48b0b330a service cinder] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:02.874909 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-3cfdc22a-bc60-4ce0-a97e-c8c48b0b330a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:02.983854 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfdc22a-bc60-4ce0-a97e-c8c48b0b330a service cinder] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:02.993216 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c447b439-95dd-4815-8164-c92c0fafaa6b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd2483f74d7d48b1a7b8ffe32f63abde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:02.994248 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447b439-95dd-4815-8164-c92c0fafaa6b service swift] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:02.994721 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447b439-95dd-4815-8164-c92c0fafaa6b service swift] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:02.997923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447b439-95dd-4815-8164-c92c0fafaa6b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:03.019780 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c447b439-95dd-4815-8164-c92c0fafaa6b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:03.021468 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c447b439-95dd-4815-8164-c92c0fafaa6b service swift] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:03.056298 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1170/2319] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Jul 26 12:54:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 12:54:04.142343 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94117d5d-2214-4541-a6c6-f4391a288a0d None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:04.142899 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94117d5d-2214-4541-a6c6-f4391a288a0d None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:04.142996 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94117d5d-2214-4541-a6c6-f4391a288a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:04.173923 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-94117d5d-2214-4541-a6c6-f4391a288a0d None None] MFA Rules not processed for user `30fc532f464843a3ad35e4d3c387bb3a`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:54:04.175624 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-94117d5d-2214-4541-a6c6-f4391a288a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:04.206252 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1150/2320] 10.222.0.20 () {58 vars in 967 bytes} [Tue Jul 26 12:54:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:54:04.208430 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c447b439-95dd-4815-8164-c92c0fafaa6b service swift] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:04.232778 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c447b439-95dd-4815-8164-c92c0fafaa6b service swift] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:04.241037 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb5da5-9394-4acd-b4bc-03ecb302a0c6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd2483f74d7d48b1a7b8ffe32f63abde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:04.241948 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb5da5-9394-4acd-b4bc-03ecb302a0c6 service swift] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:04.242406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb5da5-9394-4acd-b4bc-03ecb302a0c6 service swift] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:04.243016 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb5da5-9394-4acd-b4bc-03ecb302a0c6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:04.250416 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:54:04.253333 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eb5da5-9394-4acd-b4bc-03ecb302a0c6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:04.255969 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eb5da5-9394-4acd-b4bc-03ecb302a0c6 service swift] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:04.267190 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1171/2321] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Jul 26 12:54:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 12:54:04.703950 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a5907b-87b6-4392-9c3d-f0df6412d97b None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:04.704529 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a5907b-87b6-4392-9c3d-f0df6412d97b None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:04.705015 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a5907b-87b6-4392-9c3d-f0df6412d97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:04.738270 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-b1a5907b-87b6-4392-9c3d-f0df6412d97b None None] MFA Rules not processed for user `30fc532f464843a3ad35e4d3c387bb3a`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:54:04.740338 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-b1a5907b-87b6-4392-9c3d-f0df6412d97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:04.748145 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1151/2322] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Jul 26 12:54:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:54:04.762384 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb5da5-9394-4acd-b4bc-03ecb302a0c6 service swift] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:04.778701 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb5da5-9394-4acd-b4bc-03ecb302a0c6 service swift] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:04.793836 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce478ae-3bda-4829-8930-3db53fcc679f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd2483f74d7d48b1a7b8ffe32f63abde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:04.795153 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce478ae-3bda-4829-8930-3db53fcc679f service swift] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:04.795929 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce478ae-3bda-4829-8930-3db53fcc679f service swift] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:04.796568 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce478ae-3bda-4829-8930-3db53fcc679f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:04.807347 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce478ae-3bda-4829-8930-3db53fcc679f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:04.809149 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce478ae-3bda-4829-8930-3db53fcc679f service swift] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:04.819062 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1172/2323] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Jul 26 12:54:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 12:54:05.261414 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d3a31c-85b9-49a6-9f6e-31ef57061dab None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:05.261911 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d3a31c-85b9-49a6-9f6e-31ef57061dab None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:05.262124 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d3a31c-85b9-49a6-9f6e-31ef57061dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:05.283979 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-e8d3a31c-85b9-49a6-9f6e-31ef57061dab None None] MFA Rules not processed for user `30fc532f464843a3ad35e4d3c387bb3a`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:54:05.285302 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-e8d3a31c-85b9-49a6-9f6e-31ef57061dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:05.296934 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1152/2324] 10.222.0.20 () {58 vars in 967 bytes} [Tue Jul 26 12:54:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:54:05.337860 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce478ae-3bda-4829-8930-3db53fcc679f service swift] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:05.379064 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce478ae-3bda-4829-8930-3db53fcc679f service swift] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:05.386258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6cec1a-0375-4733-a91f-5ff92a3dd21a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd2483f74d7d48b1a7b8ffe32f63abde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:05.387185 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6cec1a-0375-4733-a91f-5ff92a3dd21a service swift] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:05.387714 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6cec1a-0375-4733-a91f-5ff92a3dd21a service swift] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:05.388216 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6cec1a-0375-4733-a91f-5ff92a3dd21a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:05.398906 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6cec1a-0375-4733-a91f-5ff92a3dd21a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:05.403253 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6cec1a-0375-4733-a91f-5ff92a3dd21a service swift] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:05.415692 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1173/2325] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Jul 26 12:54:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 12:54:08.262124 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d3a31c-85b9-49a6-9f6e-31ef57061dab None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:08.264031 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-e8d3a31c-85b9-49a6-9f6e-31ef57061dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:08.335231 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d3a31c-85b9-49a6-9f6e-31ef57061dab None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:08.339739 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b15127-76e0-41de-9700-3af5747a2865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:08.340319 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b15127-76e0-41de-9700-3af5747a2865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:08.340614 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b15127-76e0-41de-9700-3af5747a2865 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:08.340865 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b15127-76e0-41de-9700-3af5747a2865 admin admin] PATH_INFO: `/v3/users/bc8e48fa3a38429b9925b519ca28c1e2` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:08.342031 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b15127-76e0-41de-9700-3af5747a2865 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc8e48fa3a38429b9925b519ca28c1e2)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:08.344015 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b15127-76e0-41de-9700-3af5747a2865 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:08.394054 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-49b15127-76e0-41de-9700-3af5747a2865 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc8e48fa3a38429b9925b519ca28c1e2'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:54:08.394800 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-49b15127-76e0-41de-9700-3af5747a2865 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc8e48fa3a38429b9925b519ca28c1e2'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:54:08.396032 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-49b15127-76e0-41de-9700-3af5747a2865 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc8e48fa3a38429b9925b519ca28c1e2'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:54:08.408573 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-49b15127-76e0-41de-9700-3af5747a2865 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc8e48fa3a38429b9925b519ca28c1e2'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:54:08.416336 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-49b15127-76e0-41de-9700-3af5747a2865 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc8e48fa3a38429b9925b519ca28c1e2'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:54:08.433157 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1153/2326] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:54:08 2022] DELETE /identity/v3/users/bc8e48fa3a38429b9925b519ca28c1e2 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:54:08.434800 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb3beb-b43c-4e4b-9c45-577e9ebbebf6 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:08.435283 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb3beb-b43c-4e4b-9c45-577e9ebbebf6 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:08.435404 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb3beb-b43c-4e4b-9c45-577e9ebbebf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:08.451690 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b15127-76e0-41de-9700-3af5747a2865 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:08.453088 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-49b15127-76e0-41de-9700-3af5747a2865 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:08.454874 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-1ecb3beb-b43c-4e4b-9c45-577e9ebbebf6 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:54:08.456097 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-1ecb3beb-b43c-4e4b-9c45-577e9ebbebf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:08.491166 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:54:08.524322 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1174/2327] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:54:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:54:08.525924 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b15127-76e0-41de-9700-3af5747a2865 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:08.530706 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb55942-cd32-424f-a166-793b3fb40e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:08.531347 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb55942-cd32-424f-a166-793b3fb40e39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:08.531677 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb55942-cd32-424f-a166-793b3fb40e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:08.531957 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb55942-cd32-424f-a166-793b3fb40e39 admin admin] PATH_INFO: `/v3/users/fd8c9fec30ef47e3b3ca020027efb91c` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:08.533170 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb55942-cd32-424f-a166-793b3fb40e39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd8c9fec30ef47e3b3ca020027efb91c)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:08.534266 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb3beb-b43c-4e4b-9c45-577e9ebbebf6 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:08.535130 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb55942-cd32-424f-a166-793b3fb40e39 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:08.536300 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-1ecb3beb-b43c-4e4b-9c45-577e9ebbebf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:08.589307 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-cdb55942-cd32-424f-a166-793b3fb40e39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd8c9fec30ef47e3b3ca020027efb91c'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:54:08.590156 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-cdb55942-cd32-424f-a166-793b3fb40e39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd8c9fec30ef47e3b3ca020027efb91c'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:54:08.591072 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-cdb55942-cd32-424f-a166-793b3fb40e39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd8c9fec30ef47e3b3ca020027efb91c'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:54:08.596100 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb3beb-b43c-4e4b-9c45-577e9ebbebf6 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:08.600249 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4eb78b-58b8-4e93-9d3a-207d0889afb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:08.600862 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4eb78b-58b8-4e93-9d3a-207d0889afb6 service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:08.601141 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4eb78b-58b8-4e93-9d3a-207d0889afb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:08.601410 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4eb78b-58b8-4e93-9d3a-207d0889afb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:08.603497 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-9f4eb78b-58b8-4e93-9d3a-207d0889afb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:08.608147 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-cdb55942-cd32-424f-a166-793b3fb40e39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd8c9fec30ef47e3b3ca020027efb91c'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:54:08.615741 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-cdb55942-cd32-424f-a166-793b3fb40e39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd8c9fec30ef47e3b3ca020027efb91c'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:54:08.616373 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4eb78b-58b8-4e93-9d3a-207d0889afb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:08.617809 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4eb78b-58b8-4e93-9d3a-207d0889afb6 service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:08.630071 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1175/2328] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:54:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 12:54:08.630701 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1154/2329] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:54:08 2022] DELETE /identity/v3/users/fd8c9fec30ef47e3b3ca020027efb91c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:54:08.634925 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ac965-cf11-4c3b-9824-616dfdef15a1 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:08.635219 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ac965-cf11-4c3b-9824-616dfdef15a1 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:08.635498 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ac965-cf11-4c3b-9824-616dfdef15a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:08.655731 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-7c5ac965-cf11-4c3b-9824-616dfdef15a1 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:54:08.657108 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-7c5ac965-cf11-4c3b-9824-616dfdef15a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:08.736758 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1176/2330] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:54:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:54:08.749783 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb55942-cd32-424f-a166-793b3fb40e39 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:08.753606 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-cdb55942-cd32-424f-a166-793b3fb40e39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:08.825921 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb55942-cd32-424f-a166-793b3fb40e39 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:08.830344 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3104d21-973f-4b62-a787-e7b38711b579 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:08.831223 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3104d21-973f-4b62-a787-e7b38711b579 service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:08.831647 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3104d21-973f-4b62-a787-e7b38711b579 service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:08.832043 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3104d21-973f-4b62-a787-e7b38711b579 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:08.836049 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5ac965-cf11-4c3b-9824-616dfdef15a1 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:08.837658 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-7c5ac965-cf11-4c3b-9824-616dfdef15a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:08.845565 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3104d21-973f-4b62-a787-e7b38711b579 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:08.847090 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3104d21-973f-4b62-a787-e7b38711b579 service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:08.851422 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5ac965-cf11-4c3b-9824-616dfdef15a1 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:08.857782 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd857637-d401-4db2-bc81-775ffc9b991c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:08.859184 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd857637-d401-4db2-bc81-775ffc9b991c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:08.859737 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd857637-d401-4db2-bc81-775ffc9b991c admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:08.860076 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd857637-d401-4db2-bc81-775ffc9b991c admin admin] PATH_INFO: `/v3/projects/fbda6b838e3b4a8f89a693e4c5174695` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:08.861508 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd857637-d401-4db2-bc81-775ffc9b991c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbda6b838e3b4a8f89a693e4c5174695)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:08.863511 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd857637-d401-4db2-bc81-775ffc9b991c admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:08.870722 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1155/2331] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:54:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:54:08.873951 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:54:08.928639 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-fd857637-d401-4db2-bc81-775ffc9b991c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbda6b838e3b4a8f89a693e4c5174695', 'request_id': 'req-fd857637-d401-4db2-bc81-775ffc9b991c'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:54:08.937516 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-fd857637-d401-4db2-bc81-775ffc9b991c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbda6b838e3b4a8f89a693e4c5174695', 'request_id': 'req-fd857637-d401-4db2-bc81-775ffc9b991c'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:54:08.954633 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-fd857637-d401-4db2-bc81-775ffc9b991c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbda6b838e3b4a8f89a693e4c5174695', 'request_id': 'req-fd857637-d401-4db2-bc81-775ffc9b991c'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:54:08.968463 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-fd857637-d401-4db2-bc81-775ffc9b991c admin admin] The token cache is being invalidate because project fbda6b838e3b4a8f89a693e4c5174695 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:54:08.968746 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-fd857637-d401-4db2-bc81-775ffc9b991c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:54:08.969575 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-fd857637-d401-4db2-bc81-775ffc9b991c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:54:08.971270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1177/2332] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:54:08 2022] DELETE /identity/v3/projects/fbda6b838e3b4a8f89a693e4c5174695 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:54:09.242795 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3104d21-973f-4b62-a787-e7b38711b579 service neutron] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:09.244534 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-b3104d21-973f-4b62-a787-e7b38711b579 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:09.259546 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3104d21-973f-4b62-a787-e7b38711b579 service neutron] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:09.314239 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:54:09.317774 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd857637-d401-4db2-bc81-775ffc9b991c admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:09.319739 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-fd857637-d401-4db2-bc81-775ffc9b991c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:09.325958 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de83388-f3d5-47bb-82fa-4593e1024ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:09.326705 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de83388-f3d5-47bb-82fa-4593e1024ab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:09.327948 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de83388-f3d5-47bb-82fa-4593e1024ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:09.328752 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de83388-f3d5-47bb-82fa-4593e1024ab8 admin admin] PATH_INFO: `/v3/users/b8c3094e6799451dbe310dc5ddbe0155` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:09.334249 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de83388-f3d5-47bb-82fa-4593e1024ab8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8c3094e6799451dbe310dc5ddbe0155)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:09.336961 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de83388-f3d5-47bb-82fa-4593e1024ab8 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:09.340781 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd857637-d401-4db2-bc81-775ffc9b991c admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:09.348883 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9457ca-de32-4e33-8680-b9b42ba756be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:09.349680 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9457ca-de32-4e33-8680-b9b42ba756be admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:09.350029 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9457ca-de32-4e33-8680-b9b42ba756be admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:09.350420 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9457ca-de32-4e33-8680-b9b42ba756be admin admin] PATH_INFO: `/v3/projects/d04db2b45b39477296aca9dfad7ddfc4` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:09.351881 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9457ca-de32-4e33-8680-b9b42ba756be admin admin] RBAC: Authorizing `identity:delete_project(project_id=d04db2b45b39477296aca9dfad7ddfc4)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:09.354663 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9457ca-de32-4e33-8680-b9b42ba756be admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:09.396479 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-7de83388-f3d5-47bb-82fa-4593e1024ab8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8c3094e6799451dbe310dc5ddbe0155'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:54:09.397265 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-7de83388-f3d5-47bb-82fa-4593e1024ab8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8c3094e6799451dbe310dc5ddbe0155'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:54:09.397957 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-7de83388-f3d5-47bb-82fa-4593e1024ab8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8c3094e6799451dbe310dc5ddbe0155'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:54:09.414991 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-7de83388-f3d5-47bb-82fa-4593e1024ab8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8c3094e6799451dbe310dc5ddbe0155'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:54:09.420665 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-8c9457ca-de32-4e33-8680-b9b42ba756be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd04db2b45b39477296aca9dfad7ddfc4', 'request_id': 'req-8c9457ca-de32-4e33-8680-b9b42ba756be'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:54:09.422471 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-7de83388-f3d5-47bb-82fa-4593e1024ab8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8c3094e6799451dbe310dc5ddbe0155'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:54:09.428746 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-8c9457ca-de32-4e33-8680-b9b42ba756be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd04db2b45b39477296aca9dfad7ddfc4', 'request_id': 'req-8c9457ca-de32-4e33-8680-b9b42ba756be'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:54:09.437273 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-8c9457ca-de32-4e33-8680-b9b42ba756be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd04db2b45b39477296aca9dfad7ddfc4', 'request_id': 'req-8c9457ca-de32-4e33-8680-b9b42ba756be'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:54:09.439882 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1156/2333] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:54:09 2022] DELETE /identity/v3/users/b8c3094e6799451dbe310dc5ddbe0155 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:54:09.445007 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3d90e-6cbd-4b5d-96f3-deadfd1bd6ab None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:09.445007 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3d90e-6cbd-4b5d-96f3-deadfd1bd6ab None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:09.445272 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3d90e-6cbd-4b5d-96f3-deadfd1bd6ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:09.454997 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-8c9457ca-de32-4e33-8680-b9b42ba756be admin admin] The token cache is being invalidate because project d04db2b45b39477296aca9dfad7ddfc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:54:09.455319 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-8c9457ca-de32-4e33-8680-b9b42ba756be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:54:09.456435 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-8c9457ca-de32-4e33-8680-b9b42ba756be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:54:09.459031 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1178/2334] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:54:09 2022] DELETE /identity/v3/projects/d04db2b45b39477296aca9dfad7ddfc4 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:54:09.465113 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31466adb-c7f0-4970-bd24-4ceed044b64a None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:09.465441 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31466adb-c7f0-4970-bd24-4ceed044b64a None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:09.465716 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31466adb-c7f0-4970-bd24-4ceed044b64a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:09.470319 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-7ea3d90e-6cbd-4b5d-96f3-deadfd1bd6ab None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:54:09.471561 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-7ea3d90e-6cbd-4b5d-96f3-deadfd1bd6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:09.488198 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-31466adb-c7f0-4970-bd24-4ceed044b64a None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:54:09.489958 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-31466adb-c7f0-4970-bd24-4ceed044b64a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:09.554137 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1157/2335] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:54:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:54:09.565086 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1179/2336] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:54:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:54:09.568631 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea3d90e-6cbd-4b5d-96f3-deadfd1bd6ab None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:09.569176 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8c8c2-b80b-4f56-8b3f-511f5fd8d778 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:09.569452 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8c8c2-b80b-4f56-8b3f-511f5fd8d778 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:09.569737 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8c8c2-b80b-4f56-8b3f-511f5fd8d778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:09.570150 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-7ea3d90e-6cbd-4b5d-96f3-deadfd1bd6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:09.591781 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-abc8c8c2-b80b-4f56-8b3f-511f5fd8d778 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:54:09.593868 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-abc8c8c2-b80b-4f56-8b3f-511f5fd8d778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:09.603128 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1180/2337] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:54:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:54:09.607181 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc8c8c2-b80b-4f56-8b3f-511f5fd8d778 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:09.617258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc8c8c2-b80b-4f56-8b3f-511f5fd8d778 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:09.622013 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f9a73-bafb-4a24-a4c9-611fc199c314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:09.622618 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f9a73-bafb-4a24-a4c9-611fc199c314 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:09.622900 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f9a73-bafb-4a24-a4c9-611fc199c314 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:09.623175 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f9a73-bafb-4a24-a4c9-611fc199c314 admin admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:09.623908 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216f9a73-bafb-4a24-a4c9-611fc199c314 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:54:09.624212 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216f9a73-bafb-4a24-a4c9-611fc199c314 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:09.626108 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216f9a73-bafb-4a24-a4c9-611fc199c314 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:09.634256 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea3d90e-6cbd-4b5d-96f3-deadfd1bd6ab None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:09.637144 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:54:09.638747 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4039b79a-cb5f-4a51-a9f4-90ba048aaf97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:09.640306 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4039b79a-cb5f-4a51-a9f4-90ba048aaf97 service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:09.640583 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4039b79a-cb5f-4a51-a9f4-90ba048aaf97 service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:09.640882 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4039b79a-cb5f-4a51-a9f4-90ba048aaf97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:09.641663 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1181/2338] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:54:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:54:09.647547 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f9a73-bafb-4a24-a4c9-611fc199c314 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:09.651790 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4039b79a-cb5f-4a51-a9f4-90ba048aaf97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:09.653225 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4039b79a-cb5f-4a51-a9f4-90ba048aaf97 service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:09.658632 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f9a73-bafb-4a24-a4c9-611fc199c314 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:09.663469 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1158/2339] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:54:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:54:09.663887 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e69351-0eda-4ddf-8bd9-cdd846229b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:09.664487 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e69351-0eda-4ddf-8bd9-cdd846229b91 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:09.664774 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e69351-0eda-4ddf-8bd9-cdd846229b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:09.665050 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e69351-0eda-4ddf-8bd9-cdd846229b91 admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:09.665842 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e69351-0eda-4ddf-8bd9-cdd846229b91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:09.667779 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e69351-0eda-4ddf-8bd9-cdd846229b91 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:09.696426 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1182/2340] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:54:09 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:54:09.700618 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4039b79a-cb5f-4a51-a9f4-90ba048aaf97 service neutron] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:09.710887 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4039b79a-cb5f-4a51-a9f4-90ba048aaf97 service neutron] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:09.724980 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:54:09.779370 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92dde51-7d76-4d98-bc28-a9fd0a7e8bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:09.781009 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92dde51-7d76-4d98-bc28-a9fd0a7e8bb9 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:09.782210 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92dde51-7d76-4d98-bc28-a9fd0a7e8bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:09.782705 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92dde51-7d76-4d98-bc28-a9fd0a7e8bb9 admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:09.784273 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92dde51-7d76-4d98-bc28-a9fd0a7e8bb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:09.789137 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92dde51-7d76-4d98-bc28-a9fd0a7e8bb9 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:09.824623 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1159/2341] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:54:09 2022] POST /identity/v3/users => generated 286 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:54:09.829008 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e69351-0eda-4ddf-8bd9-cdd846229b91 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:09.844305 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e69351-0eda-4ddf-8bd9-cdd846229b91 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:09.850309 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a0fb2d-9808-4315-a0f1-21f4028637ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:09.850943 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0fb2d-9808-4315-a0f1-21f4028637ee admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:09.851226 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0fb2d-9808-4315-a0f1-21f4028637ee admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:09.851529 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0fb2d-9808-4315-a0f1-21f4028637ee admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:09.852289 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a0fb2d-9808-4315-a0f1-21f4028637ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:09.854019 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a0fb2d-9808-4315-a0f1-21f4028637ee admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:09.864730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1183/2342] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:54:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:54:09.869769 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92dde51-7d76-4d98-bc28-a9fd0a7e8bb9 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:09.881405 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92dde51-7d76-4d98-bc28-a9fd0a7e8bb9 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:09.888144 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d8ebc8-db75-400b-8d55-c2cf9059b2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:09.888869 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8ebc8-db75-400b-8d55-c2cf9059b2e4 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:09.889699 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8ebc8-db75-400b-8d55-c2cf9059b2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:09.890048 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8ebc8-db75-400b-8d55-c2cf9059b2e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:09.890837 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d8ebc8-db75-400b-8d55-c2cf9059b2e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:09.892653 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d8ebc8-db75-400b-8d55-c2cf9059b2e4 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:09.905974 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1160/2343] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:54:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:54:09.910107 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a0fb2d-9808-4315-a0f1-21f4028637ee admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:09.921370 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a0fb2d-9808-4315-a0f1-21f4028637ee admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:09.926970 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0399aa55-c611-4a43-ab25-d7bab582a7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:09.927693 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0399aa55-c611-4a43-ab25-d7bab582a7aa admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:09.927968 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0399aa55-c611-4a43-ab25-d7bab582a7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:09.928247 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0399aa55-c611-4a43-ab25-d7bab582a7aa admin admin] PATH_INFO: `/v3/projects/b8383c3af34e46f2922ffb7f6d4963af/users/b6c556056d054689ab7b100f02404d3f/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:09.946200 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0399aa55-c611-4a43-ab25-d7bab582a7aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8383c3af34e46f2922ffb7f6d4963af, user_id=b6c556056d054689ab7b100f02404d3f, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:09.949121 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0399aa55-c611-4a43-ab25-d7bab582a7aa admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:09.981628 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1184/2344] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:54:09 2022] PUT /identity/v3/projects/b8383c3af34e46f2922ffb7f6d4963af/users/b6c556056d054689ab7b100f02404d3f/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:54:09.988957 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd935d80-86d8-41ab-8b24-5501411df247 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:09.989391 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd935d80-86d8-41ab-8b24-5501411df247 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:09.989980 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd935d80-86d8-41ab-8b24-5501411df247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:10.029508 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-fd935d80-86d8-41ab-8b24-5501411df247 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:54:10.032033 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-fd935d80-86d8-41ab-8b24-5501411df247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:10.055876 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0399aa55-c611-4a43-ab25-d7bab582a7aa admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:10.058775 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-0399aa55-c611-4a43-ab25-d7bab582a7aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:10.136336 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1161/2345] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:54:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:54:10.147020 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd935d80-86d8-41ab-8b24-5501411df247 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:10.148222 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0399aa55-c611-4a43-ab25-d7bab582a7aa admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:10.148617 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-fd935d80-86d8-41ab-8b24-5501411df247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:10.153095 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d643a2e-7df3-4a40-91d2-57393d426b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:10.153748 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d643a2e-7df3-4a40-91d2-57393d426b9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:10.154032 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d643a2e-7df3-4a40-91d2-57393d426b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:10.154312 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d643a2e-7df3-4a40-91d2-57393d426b9a admin admin] PATH_INFO: `/v3/projects/ca9be6b3f09745ac84a69ea4d4b9cc79` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:10.155520 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d643a2e-7df3-4a40-91d2-57393d426b9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca9be6b3f09745ac84a69ea4d4b9cc79)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:10.157496 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d643a2e-7df3-4a40-91d2-57393d426b9a admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:10.208368 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd935d80-86d8-41ab-8b24-5501411df247 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:10.214928 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-1d643a2e-7df3-4a40-91d2-57393d426b9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca9be6b3f09745ac84a69ea4d4b9cc79', 'request_id': 'req-1d643a2e-7df3-4a40-91d2-57393d426b9a'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:54:10.222324 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-1d643a2e-7df3-4a40-91d2-57393d426b9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca9be6b3f09745ac84a69ea4d4b9cc79', 'request_id': 'req-1d643a2e-7df3-4a40-91d2-57393d426b9a'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:54:10.229870 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-1d643a2e-7df3-4a40-91d2-57393d426b9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca9be6b3f09745ac84a69ea4d4b9cc79', 'request_id': 'req-1d643a2e-7df3-4a40-91d2-57393d426b9a'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:54:10.241248 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-1d643a2e-7df3-4a40-91d2-57393d426b9a admin admin] The token cache is being invalidate because project ca9be6b3f09745ac84a69ea4d4b9cc79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:54:10.241539 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-1d643a2e-7df3-4a40-91d2-57393d426b9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:54:10.242201 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-1d643a2e-7df3-4a40-91d2-57393d426b9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:54:10.243946 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1185/2346] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:54:10 2022] DELETE /identity/v3/projects/ca9be6b3f09745ac84a69ea4d4b9cc79 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:54:10.251159 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628680e-b0ca-4c6f-934c-2d81d60be143 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:10.251450 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628680e-b0ca-4c6f-934c-2d81d60be143 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:10.251759 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628680e-b0ca-4c6f-934c-2d81d60be143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:10.262819 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61a8dda-f676-42d7-ad58-274778cd9993 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:10.263454 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a8dda-f676-42d7-ad58-274778cd9993 service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:10.263765 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a8dda-f676-42d7-ad58-274778cd9993 service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:10.264227 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a8dda-f676-42d7-ad58-274778cd9993 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:10.266108 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-e61a8dda-f676-42d7-ad58-274778cd9993 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:10.273456 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-5628680e-b0ca-4c6f-934c-2d81d60be143 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:54:10.273670 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:54:10.274664 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-5628680e-b0ca-4c6f-934c-2d81d60be143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:10.310999 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:54:10.341008 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61a8dda-f676-42d7-ad58-274778cd9993 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:10.342471 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61a8dda-f676-42d7-ad58-274778cd9993 service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:10.345624 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1186/2347] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:54:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:54:10.353007 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1162/2348] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:54:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 12:54:10.354398 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b023ee-ee8d-4da7-aaed-9b32839636d4 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:10.354761 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b023ee-ee8d-4da7-aaed-9b32839636d4 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:10.354983 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b023ee-ee8d-4da7-aaed-9b32839636d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:10.378505 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-f0b023ee-ee8d-4da7-aaed-9b32839636d4 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:54:10.380376 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-f0b023ee-ee8d-4da7-aaed-9b32839636d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:10.389275 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1187/2349] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:54:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:54:10.393528 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61a8dda-f676-42d7-ad58-274778cd9993 service neutron] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:10.404034 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61a8dda-f676-42d7-ad58-274778cd9993 service neutron] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:10.409263 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfdc476-3b87-4f05-817f-c38f8529eaf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:10.409620 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfdc476-3b87-4f05-817f-c38f8529eaf7 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:10.409965 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfdc476-3b87-4f05-817f-c38f8529eaf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:10.410361 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfdc476-3b87-4f05-817f-c38f8529eaf7 admin admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:10.411177 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfdc476-3b87-4f05-817f-c38f8529eaf7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:54:10.411552 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfdc476-3b87-4f05-817f-c38f8529eaf7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:10.413283 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfdc476-3b87-4f05-817f-c38f8529eaf7 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:10.426102 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1163/2350] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:54:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:54:10.431138 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b023ee-ee8d-4da7-aaed-9b32839636d4 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:10.441636 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b023ee-ee8d-4da7-aaed-9b32839636d4 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:10.447520 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e5db4-def1-4185-a1e4-0ab95fee3158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:10.448241 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e5db4-def1-4185-a1e4-0ab95fee3158 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:10.448536 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e5db4-def1-4185-a1e4-0ab95fee3158 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:10.448832 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e5db4-def1-4185-a1e4-0ab95fee3158 admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:10.449549 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510e5db4-def1-4185-a1e4-0ab95fee3158 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:10.451508 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510e5db4-def1-4185-a1e4-0ab95fee3158 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:10.481543 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1188/2351] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:54:10 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:54:10.485878 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfdc476-3b87-4f05-817f-c38f8529eaf7 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:10.496866 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfdc476-3b87-4f05-817f-c38f8529eaf7 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:10.565880 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ad2fa-fce8-456d-8ee5-f86006b5c784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:10.566501 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ad2fa-fce8-456d-8ee5-f86006b5c784 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:10.566804 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ad2fa-fce8-456d-8ee5-f86006b5c784 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:10.567715 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ad2fa-fce8-456d-8ee5-f86006b5c784 admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:10.568432 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265ad2fa-fce8-456d-8ee5-f86006b5c784 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:10.570737 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265ad2fa-fce8-456d-8ee5-f86006b5c784 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:10.595694 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1164/2352] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:54:10 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:54:10.599746 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e5db4-def1-4185-a1e4-0ab95fee3158 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:10.611101 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e5db4-def1-4185-a1e4-0ab95fee3158 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:10.616712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2108aa-5325-4f26-96fd-12fc63905a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:10.617788 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2108aa-5325-4f26-96fd-12fc63905a12 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:10.617788 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2108aa-5325-4f26-96fd-12fc63905a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:10.618042 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2108aa-5325-4f26-96fd-12fc63905a12 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:10.618715 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2108aa-5325-4f26-96fd-12fc63905a12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:10.620338 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2108aa-5325-4f26-96fd-12fc63905a12 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:10.635782 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1189/2353] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:54:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:54:10.637629 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ad2fa-fce8-456d-8ee5-f86006b5c784 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:10.648322 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ad2fa-fce8-456d-8ee5-f86006b5c784 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:10.653128 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c486f-c246-4939-8809-80b8227eee02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:10.653734 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c486f-c246-4939-8809-80b8227eee02 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:10.654003 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c486f-c246-4939-8809-80b8227eee02 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:10.654274 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c486f-c246-4939-8809-80b8227eee02 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:10.654968 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218c486f-c246-4939-8809-80b8227eee02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:10.656596 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218c486f-c246-4939-8809-80b8227eee02 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:10.671686 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1165/2354] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:54:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:54:10.676761 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2108aa-5325-4f26-96fd-12fc63905a12 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:10.687432 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2108aa-5325-4f26-96fd-12fc63905a12 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:10.692903 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572119f9-cbcc-4dd3-aa9a-12748c63be6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:10.693659 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572119f9-cbcc-4dd3-aa9a-12748c63be6c admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:10.694041 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572119f9-cbcc-4dd3-aa9a-12748c63be6c admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:10.694849 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572119f9-cbcc-4dd3-aa9a-12748c63be6c admin admin] PATH_INFO: `/v3/projects/a2522985c6df452dabe2010e5f6bed35/users/94efba410653451da2e9977d746633f6/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:10.715120 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572119f9-cbcc-4dd3-aa9a-12748c63be6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2522985c6df452dabe2010e5f6bed35, user_id=94efba410653451da2e9977d746633f6, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:10.718490 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572119f9-cbcc-4dd3-aa9a-12748c63be6c admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:10.742555 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1190/2355] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:54:10 2022] PUT /identity/v3/projects/a2522985c6df452dabe2010e5f6bed35/users/94efba410653451da2e9977d746633f6/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:54:10.744469 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eecd8b9-51be-4270-abe1-456b63229020 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:10.744848 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eecd8b9-51be-4270-abe1-456b63229020 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:10.745123 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eecd8b9-51be-4270-abe1-456b63229020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:10.777632 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-9eecd8b9-51be-4270-abe1-456b63229020 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:54:10.779881 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-9eecd8b9-51be-4270-abe1-456b63229020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:10.860165 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1166/2356] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:54:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:54:10.868128 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572119f9-cbcc-4dd3-aa9a-12748c63be6c admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:10.869844 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-572119f9-cbcc-4dd3-aa9a-12748c63be6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:10.944151 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572119f9-cbcc-4dd3-aa9a-12748c63be6c admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:10.947848 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67cb39f-f6f8-4281-be1d-cf2bee27c4bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:10.948488 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67cb39f-f6f8-4281-be1d-cf2bee27c4bf service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:10.948758 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67cb39f-f6f8-4281-be1d-cf2bee27c4bf service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:10.949028 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67cb39f-f6f8-4281-be1d-cf2bee27c4bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:10.959308 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67cb39f-f6f8-4281-be1d-cf2bee27c4bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:10.960819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67cb39f-f6f8-4281-be1d-cf2bee27c4bf service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:10.976310 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1191/2357] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:54:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:54:14.217843 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23bd9b3-1b25-4098-a3bc-d086b4374ce7 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:14.218398 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23bd9b3-1b25-4098-a3bc-d086b4374ce7 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:14.218398 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23bd9b3-1b25-4098-a3bc-d086b4374ce7 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:14.220183 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1167/2358] 10.222.0.20 () {54 vars in 921 bytes} [Tue Jul 26 12:54:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:54:14.231096 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fed4f8-2934-407a-b74a-66ee40d71762 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:14.231545 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fed4f8-2934-407a-b74a-66ee40d71762 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:14.231660 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fed4f8-2934-407a-b74a-66ee40d71762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:14.276757 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-c4fed4f8-2934-407a-b74a-66ee40d71762 None None] MFA Rules not processed for user `0f5d6f3c12244c0faf26048513dc49e7`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:54:14.277928 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-c4fed4f8-2934-407a-b74a-66ee40d71762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:14.398177 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1192/2359] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Jul 26 12:54:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:54:14.410934 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23bd9b3-1b25-4098-a3bc-d086b4374ce7 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:14.424742 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-e23bd9b3-1b25-4098-a3bc-d086b4374ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:14.448298 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:54:14.523601 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23bd9b3-1b25-4098-a3bc-d086b4374ce7 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:14.537645 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030a7971-7b67-4311-a055-3dce040020b2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2b5853838dc40db89c46038c0606107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:14.538662 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a7971-7b67-4311-a055-3dce040020b2 service glance] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:14.539580 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a7971-7b67-4311-a055-3dce040020b2 service glance] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:14.539994 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a7971-7b67-4311-a055-3dce040020b2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:14.554520 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030a7971-7b67-4311-a055-3dce040020b2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:14.556901 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030a7971-7b67-4311-a055-3dce040020b2 service glance] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:14.572657 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1168/2360] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:54:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:54:14.724187 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb60084-5e73-4f64-852e-c067e308eb32 None None] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:14.724615 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb60084-5e73-4f64-852e-c067e308eb32 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:14.725502 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb60084-5e73-4f64-852e-c067e308eb32 None None] PATH_INFO: `/` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:14.729000 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1193/2361] 10.222.0.20 () {54 vars in 921 bytes} [Tue Jul 26 12:54:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:54:14.745500 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebb68f-9b0e-485e-a851-48ec804803c2 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:14.746168 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebb68f-9b0e-485e-a851-48ec804803c2 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:14.746664 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebb68f-9b0e-485e-a851-48ec804803c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:14.785838 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-75ebb68f-9b0e-485e-a851-48ec804803c2 None None] MFA Rules not processed for user `6830350c6eba4610bbe9ab1dc31f5939`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:54:14.787577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-75ebb68f-9b0e-485e-a851-48ec804803c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:14.809558 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1169/2362] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Jul 26 12:54:14 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:54:14.827543 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb60084-5e73-4f64-852e-c067e308eb32 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:14.830355 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-0eb60084-5e73-4f64-852e-c067e308eb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:14.842097 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:54:14.850381 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb60084-5e73-4f64-852e-c067e308eb32 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:14.859200 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a5a1da-5359-472f-91cf-5d25cdd05cff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:14.860918 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5a1da-5359-472f-91cf-5d25cdd05cff service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:14.862107 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5a1da-5359-472f-91cf-5d25cdd05cff service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:14.862607 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5a1da-5359-472f-91cf-5d25cdd05cff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:14.883480 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a5a1da-5359-472f-91cf-5d25cdd05cff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:14.889547 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a5a1da-5359-472f-91cf-5d25cdd05cff service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:14.907032 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1194/2363] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:54:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:54:15.121969 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee0ff5-e05a-4bba-af4d-af725ca5f097 None None] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:15.122348 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee0ff5-e05a-4bba-af4d-af725ca5f097 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:15.122912 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee0ff5-e05a-4bba-af4d-af725ca5f097 None None] PATH_INFO: `/` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:15.124854 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1170/2364] 10.222.0.20 () {54 vars in 921 bytes} [Tue Jul 26 12:54:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 12:54:15.146835 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c0cd5-2e83-4496-b5f1-bfe80f7524d5 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:15.147113 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c0cd5-2e83-4496-b5f1-bfe80f7524d5 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:15.147458 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c0cd5-2e83-4496-b5f1-bfe80f7524d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:15.223727 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-1c7c0cd5-2e83-4496-b5f1-bfe80f7524d5 None None] MFA Rules not processed for user `0f5d6f3c12244c0faf26048513dc49e7`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:54:15.225305 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-1c7c0cd5-2e83-4496-b5f1-bfe80f7524d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:15.245086 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1195/2365] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Jul 26 12:54:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:54:15.293515 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee0ff5-e05a-4bba-af4d-af725ca5f097 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:15.312765 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee0ff5-e05a-4bba-af4d-af725ca5f097 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:15.339969 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603f8812-49f0-4639-b533-5cb340222fbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:15.343577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f8812-49f0-4639-b533-5cb340222fbf service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:15.343917 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f8812-49f0-4639-b533-5cb340222fbf service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:15.344196 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f8812-49f0-4639-b533-5cb340222fbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:15.364670 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603f8812-49f0-4639-b533-5cb340222fbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:15.366145 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603f8812-49f0-4639-b533-5cb340222fbf service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:15.404856 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1171/2366] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:54:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:54:16.918746 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad02c9-6c9e-4eaa-aa57-39c5cbce51c1 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:16.919618 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad02c9-6c9e-4eaa-aa57-39c5cbce51c1 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:16.920648 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad02c9-6c9e-4eaa-aa57-39c5cbce51c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:17.009119 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-ebad02c9-6c9e-4eaa-aa57-39c5cbce51c1 None None] MFA Rules not processed for user `b6c556056d054689ab7b100f02404d3f`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:54:17.010594 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-ebad02c9-6c9e-4eaa-aa57-39c5cbce51c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:17.195398 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1196/2367] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:54:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:54:17.200100 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603f8812-49f0-4639-b533-5cb340222fbf service neutron] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:17.203919 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-603f8812-49f0-4639-b533-5cb340222fbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:17.233854 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603f8812-49f0-4639-b533-5cb340222fbf service neutron] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:17.240284 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c0e14-8942-4419-9088-7fdff5abb006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:17.241398 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c0e14-8942-4419-9088-7fdff5abb006 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:17.241837 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c0e14-8942-4419-9088-7fdff5abb006 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:17.242467 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c0e14-8942-4419-9088-7fdff5abb006 admin admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:17.243515 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c0e14-8942-4419-9088-7fdff5abb006 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:17.245810 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c0e14-8942-4419-9088-7fdff5abb006 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:17.363677 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebad02c9-6c9e-4eaa-aa57-39c5cbce51c1 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:17.375660 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1172/2368] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:54:17 2022] POST /identity/v3/projects => generated 373 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:54:17.421619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebad02c9-6c9e-4eaa-aa57-39c5cbce51c1 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:17.590143 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c6f8c-f668-4ce0-aa8e-c68b8247d412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:17.591440 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c6f8c-f668-4ce0-aa8e-c68b8247d412 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:17.591810 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c6f8c-f668-4ce0-aa8e-c68b8247d412 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:17.592085 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c6f8c-f668-4ce0-aa8e-c68b8247d412 admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:17.592775 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62c6f8c-f668-4ce0-aa8e-c68b8247d412 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:17.594808 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62c6f8c-f668-4ce0-aa8e-c68b8247d412 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:17.633887 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1197/2369] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:54:17 2022] POST /identity/v3/users => generated 293 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:54:17.637952 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c0e14-8942-4419-9088-7fdff5abb006 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:17.664418 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c0e14-8942-4419-9088-7fdff5abb006 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:17.679009 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e70bd3-e01d-4d9e-b7cc-a0096e4dcd69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:17.679946 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e70bd3-e01d-4d9e-b7cc-a0096e4dcd69 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:17.680831 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e70bd3-e01d-4d9e-b7cc-a0096e4dcd69 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:17.681296 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e70bd3-e01d-4d9e-b7cc-a0096e4dcd69 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:17.682168 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e70bd3-e01d-4d9e-b7cc-a0096e4dcd69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:17.684018 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e70bd3-e01d-4d9e-b7cc-a0096e4dcd69 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:17.708705 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1173/2370] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:54:17 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:54:17.711866 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c6f8c-f668-4ce0-aa8e-c68b8247d412 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:17.724428 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c6f8c-f668-4ce0-aa8e-c68b8247d412 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:17.731650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c882ff0-f9d2-4f78-a3a8-647bacb916ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:17.732768 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c882ff0-f9d2-4f78-a3a8-647bacb916ed admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:17.734259 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c882ff0-f9d2-4f78-a3a8-647bacb916ed admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:17.734672 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c882ff0-f9d2-4f78-a3a8-647bacb916ed admin admin] PATH_INFO: `/v3/projects/fd4ed91d9355467b98e8bb224ec9d05e/users/8639ac065f9d4ae5b3a9bd771bfbd045/roles/0e78444e43c744668178971af4087ca0` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:17.760741 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c882ff0-f9d2-4f78-a3a8-647bacb916ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd4ed91d9355467b98e8bb224ec9d05e, user_id=8639ac065f9d4ae5b3a9bd771bfbd045, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:17.764908 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c882ff0-f9d2-4f78-a3a8-647bacb916ed admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:17.788861 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1198/2371] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:54:17 2022] PUT /identity/v3/projects/fd4ed91d9355467b98e8bb224ec9d05e/users/8639ac065f9d4ae5b3a9bd771bfbd045/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:54:17.793271 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e70bd3-e01d-4d9e-b7cc-a0096e4dcd69 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:17.807594 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e70bd3-e01d-4d9e-b7cc-a0096e4dcd69 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:17.947789 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b34772f-79f2-4882-b321-9ef80163a502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:17.948091 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34772f-79f2-4882-b321-9ef80163a502 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:17.948378 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34772f-79f2-4882-b321-9ef80163a502 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:17.948555 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34772f-79f2-4882-b321-9ef80163a502 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:17.951000 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b34772f-79f2-4882-b321-9ef80163a502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:17.952999 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b34772f-79f2-4882-b321-9ef80163a502 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:17.980239 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1174/2372] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:54:17 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:54:17.984705 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c882ff0-f9d2-4f78-a3a8-647bacb916ed admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:18.002328 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c882ff0-f9d2-4f78-a3a8-647bacb916ed admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:18.011130 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d89f4-77b8-4d08-a0ed-601c4fb92a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:18.012358 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d89f4-77b8-4d08-a0ed-601c4fb92a9b admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:18.012683 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d89f4-77b8-4d08-a0ed-601c4fb92a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:18.013619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d89f4-77b8-4d08-a0ed-601c4fb92a9b admin admin] PATH_INFO: `/v3/projects/fd4ed91d9355467b98e8bb224ec9d05e/users/8639ac065f9d4ae5b3a9bd771bfbd045/roles/0e78444e43c744668178971af4087ca0` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:18.018793 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7d89f4-77b8-4d08-a0ed-601c4fb92a9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd4ed91d9355467b98e8bb224ec9d05e, user_id=8639ac065f9d4ae5b3a9bd771bfbd045, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:18.025465 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7d89f4-77b8-4d08-a0ed-601c4fb92a9b admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:18.054825 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1199/2373] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:54:17 2022] PUT /identity/v3/projects/fd4ed91d9355467b98e8bb224ec9d05e/users/8639ac065f9d4ae5b3a9bd771bfbd045/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:54:18.178161 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebdcd5e-46d6-4a4a-a742-24025c773ea7 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:18.179600 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebdcd5e-46d6-4a4a-a742-24025c773ea7 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:18.180112 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebdcd5e-46d6-4a4a-a742-24025c773ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:18.227002 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-6ebdcd5e-46d6-4a4a-a742-24025c773ea7 None None] MFA Rules not processed for user `94efba410653451da2e9977d746633f6`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:54:18.228398 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-6ebdcd5e-46d6-4a4a-a742-24025c773ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:18.299344 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:54:18.334043 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1175/2374] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:54:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:54:18.337328 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d89f4-77b8-4d08-a0ed-601c4fb92a9b admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:18.378086 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:54:18.420304 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d89f4-77b8-4d08-a0ed-601c4fb92a9b admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:18.426078 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7c3ff-f6e8-4694-a4f0-1134a370b3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:18.426732 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7c3ff-f6e8-4694-a4f0-1134a370b3c9 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:18.427016 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7c3ff-f6e8-4694-a4f0-1134a370b3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:18.427280 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7c3ff-f6e8-4694-a4f0-1134a370b3c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:18.427996 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e7c3ff-f6e8-4694-a4f0-1134a370b3c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:18.429973 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e7c3ff-f6e8-4694-a4f0-1134a370b3c9 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:18.466007 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1200/2375] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:54:18 2022] POST /identity/v3/projects => generated 379 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:54:18.470134 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebdcd5e-46d6-4a4a-a742-24025c773ea7 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:18.484944 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebdcd5e-46d6-4a4a-a742-24025c773ea7 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:18.573712 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a4504-e317-4a71-ab27-4924bd15fcd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:18.574614 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a4504-e317-4a71-ab27-4924bd15fcd0 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:18.575084 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a4504-e317-4a71-ab27-4924bd15fcd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:18.575569 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a4504-e317-4a71-ab27-4924bd15fcd0 admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:18.576619 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933a4504-e317-4a71-ab27-4924bd15fcd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:18.579567 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933a4504-e317-4a71-ab27-4924bd15fcd0 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:18.610372 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1176/2376] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:54:18 2022] POST /identity/v3/users => generated 296 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:54:18.614324 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7c3ff-f6e8-4694-a4f0-1134a370b3c9 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:18.627021 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7c3ff-f6e8-4694-a4f0-1134a370b3c9 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:18.635064 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fea8b5-f5b9-446c-ad12-32e8926fb026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:18.636143 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fea8b5-f5b9-446c-ad12-32e8926fb026 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:18.636426 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fea8b5-f5b9-446c-ad12-32e8926fb026 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:18.636705 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fea8b5-f5b9-446c-ad12-32e8926fb026 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:18.637423 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fea8b5-f5b9-446c-ad12-32e8926fb026 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:18.639239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fea8b5-f5b9-446c-ad12-32e8926fb026 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:18.671370 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1201/2377] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:54:18.676673 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a4504-e317-4a71-ab27-4924bd15fcd0 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:18.695790 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a4504-e317-4a71-ab27-4924bd15fcd0 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:18.700786 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b0935-945e-4f70-a0b9-ad3aa7586bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:18.701551 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b0935-945e-4f70-a0b9-ad3aa7586bd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:18.701929 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b0935-945e-4f70-a0b9-ad3aa7586bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:18.702362 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b0935-945e-4f70-a0b9-ad3aa7586bd2 admin admin] PATH_INFO: `/v3/projects/3b53d8ebad9843729c9d9d8c5a83c2db/users/82223cea11be4790a233d1419c5e37b9/roles/0e78444e43c744668178971af4087ca0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:18.720118 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291b0935-945e-4f70-a0b9-ad3aa7586bd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b53d8ebad9843729c9d9d8c5a83c2db, user_id=82223cea11be4790a233d1419c5e37b9, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:18.723243 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291b0935-945e-4f70-a0b9-ad3aa7586bd2 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:18.746928 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1177/2378] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:54:18 2022] PUT /identity/v3/projects/3b53d8ebad9843729c9d9d8c5a83c2db/users/82223cea11be4790a233d1419c5e37b9/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:54:18.750959 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fea8b5-f5b9-446c-ad12-32e8926fb026 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:18.765468 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fea8b5-f5b9-446c-ad12-32e8926fb026 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:18.853524 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caaa5bd-2f1f-4989-a68b-cccd18cb77af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:18.855773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caaa5bd-2f1f-4989-a68b-cccd18cb77af admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:18.855773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caaa5bd-2f1f-4989-a68b-cccd18cb77af admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:18.855773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caaa5bd-2f1f-4989-a68b-cccd18cb77af admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:18.855773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caaa5bd-2f1f-4989-a68b-cccd18cb77af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:18.857788 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caaa5bd-2f1f-4989-a68b-cccd18cb77af admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:18.875303 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1202/2379] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:54:18.879149 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b0935-945e-4f70-a0b9-ad3aa7586bd2 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:18.891647 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b0935-945e-4f70-a0b9-ad3aa7586bd2 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:18.897196 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8a3621-d159-4d14-98b7-1076ae99b203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:18.897885 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a3621-d159-4d14-98b7-1076ae99b203 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:18.898213 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a3621-d159-4d14-98b7-1076ae99b203 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:18.898500 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a3621-d159-4d14-98b7-1076ae99b203 admin admin] PATH_INFO: `/v3/projects/3b53d8ebad9843729c9d9d8c5a83c2db/users/82223cea11be4790a233d1419c5e37b9/roles/0e78444e43c744668178971af4087ca0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:18.901126 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8a3621-d159-4d14-98b7-1076ae99b203 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b53d8ebad9843729c9d9d8c5a83c2db, user_id=82223cea11be4790a233d1419c5e37b9, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:18.903991 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8a3621-d159-4d14-98b7-1076ae99b203 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:18.917655 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1178/2380] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:54:18 2022] PUT /identity/v3/projects/3b53d8ebad9843729c9d9d8c5a83c2db/users/82223cea11be4790a233d1419c5e37b9/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:54:24.795109 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caaa5bd-2f1f-4989-a68b-cccd18cb77af admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:24.797482 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-6caaa5bd-2f1f-4989-a68b-cccd18cb77af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:24.879243 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8a3621-d159-4d14-98b7-1076ae99b203 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:24.881400 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-8e8a3621-d159-4d14-98b7-1076ae99b203 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:25.102609 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caaa5bd-2f1f-4989-a68b-cccd18cb77af admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:25.173224 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8a3621-d159-4d14-98b7-1076ae99b203 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:25.186947 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d81188-e7cc-4755-bf7b-00ad60a2173f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec49a03aed774c439459883bd5442d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:25.187704 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d81188-e7cc-4755-bf7b-00ad60a2173f service placement] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:25.188088 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d81188-e7cc-4755-bf7b-00ad60a2173f service placement] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:25.188460 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d81188-e7cc-4755-bf7b-00ad60a2173f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:25.192846 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-83d81188-e7cc-4755-bf7b-00ad60a2173f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:25.211422 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:54:25.217190 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d81188-e7cc-4755-bf7b-00ad60a2173f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:25.220712 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d81188-e7cc-4755-bf7b-00ad60a2173f service placement] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:25.238912 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1179/2381] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:54:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:54:25.258091 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c40be2c-8380-4f7c-9be5-36e85fe60380 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec49a03aed774c439459883bd5442d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:25.258958 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c40be2c-8380-4f7c-9be5-36e85fe60380 service placement] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:25.259338 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c40be2c-8380-4f7c-9be5-36e85fe60380 service placement] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:25.259804 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c40be2c-8380-4f7c-9be5-36e85fe60380 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:25.274515 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c40be2c-8380-4f7c-9be5-36e85fe60380 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:25.276739 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c40be2c-8380-4f7c-9be5-36e85fe60380 service placement] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:25.308339 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1203/2382] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:54:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:54:26.977386 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba564ecb-772c-47a3-8286-d4432e826ba9 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:26.978049 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba564ecb-772c-47a3-8286-d4432e826ba9 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:26.978049 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba564ecb-772c-47a3-8286-d4432e826ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:27.047521 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-ba564ecb-772c-47a3-8286-d4432e826ba9 None None] MFA Rules not processed for user `8639ac065f9d4ae5b3a9bd771bfbd045`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:54:27.048945 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-ba564ecb-772c-47a3-8286-d4432e826ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:27.085640 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:54:27.223158 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1180/2383] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:54:26 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:54:27.233275 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c40be2c-8380-4f7c-9be5-36e85fe60380 service placement] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:27.238983 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-5c40be2c-8380-4f7c-9be5-36e85fe60380 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:27.395161 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c40be2c-8380-4f7c-9be5-36e85fe60380 service placement] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:27.412316 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b6c392-c2d4-4d21-98a6-be0b617a4b97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:27.413713 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6c392-c2d4-4d21-98a6-be0b617a4b97 service nova] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:27.415833 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6c392-c2d4-4d21-98a6-be0b617a4b97 service nova] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:27.416296 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6c392-c2d4-4d21-98a6-be0b617a4b97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:27.530438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18930657-780e-42c2-81ad-c318c393ce4c None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:27.530737 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18930657-780e-42c2-81ad-c318c393ce4c None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:27.531058 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18930657-780e-42c2-81ad-c318c393ce4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:27.537282 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b6c392-c2d4-4d21-98a6-be0b617a4b97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:27.540378 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b6c392-c2d4-4d21-98a6-be0b617a4b97 service nova] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:27.568688 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1204/2384] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:54:27 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:54:27.581804 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-18930657-780e-42c2-81ad-c318c393ce4c None None] MFA Rules not processed for user `82223cea11be4790a233d1419c5e37b9`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:54:27.583562 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-18930657-780e-42c2-81ad-c318c393ce4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:27.665307 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1181/2385] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:54:27 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:54:27.687970 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b6c392-c2d4-4d21-98a6-be0b617a4b97 service nova] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:27.689914 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-80b6c392-c2d4-4d21-98a6-be0b617a4b97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:27.746567 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b6c392-c2d4-4d21-98a6-be0b617a4b97 service nova] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:27.766600 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4245faf7-580f-4db3-bb32-ab0bde4184a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:27.766801 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245faf7-580f-4db3-bb32-ab0bde4184a3 service nova] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:27.767113 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245faf7-580f-4db3-bb32-ab0bde4184a3 service nova] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:27.767389 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245faf7-580f-4db3-bb32-ab0bde4184a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:27.865523 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4245faf7-580f-4db3-bb32-ab0bde4184a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:27.867907 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4245faf7-580f-4db3-bb32-ab0bde4184a3 service nova] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:27.884348 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1205/2386] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:54:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 12:54:29.834050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b4f65-d9eb-42ae-bb0c-b9814082b527 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:29.834565 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b4f65-d9eb-42ae-bb0c-b9814082b527 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:29.834565 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b4f65-d9eb-42ae-bb0c-b9814082b527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:29.854476 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-af8b4f65-d9eb-42ae-bb0c-b9814082b527 None None] MFA Rules not processed for user `b6c556056d054689ab7b100f02404d3f`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:54:29.855772 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-af8b4f65-d9eb-42ae-bb0c-b9814082b527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:29.869747 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1182/2387] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:54:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:54:37.878735 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=98 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:54:39.848679 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=59 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:54:49.527184 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a828848-4114-4d5f-bed9-ef58e497a016 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:49.527794 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a828848-4114-4d5f-bed9-ef58e497a016 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:49.527794 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a828848-4114-4d5f-bed9-ef58e497a016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:49.548272 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-7a828848-4114-4d5f-bed9-ef58e497a016 None None] MFA Rules not processed for user `30fc532f464843a3ad35e4d3c387bb3a`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:54:49.549641 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-7a828848-4114-4d5f-bed9-ef58e497a016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:49.605398 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1206/2388] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Jul 26 12:54:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:54:49.617766 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8b4f65-d9eb-42ae-bb0c-b9814082b527 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:54:49.619170 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-af8b4f65-d9eb-42ae-bb0c-b9814082b527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:49.690555 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8b4f65-d9eb-42ae-bb0c-b9814082b527 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:54:49.696327 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bc8dd5-60a7-4d13-90fd-6860fa933e26 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd2483f74d7d48b1a7b8ffe32f63abde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:54:49.697066 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc8dd5-60a7-4d13-90fd-6860fa933e26 service swift] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:49.697356 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc8dd5-60a7-4d13-90fd-6860fa933e26 service swift] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:49.697632 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc8dd5-60a7-4d13-90fd-6860fa933e26 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:49.705818 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bc8dd5-60a7-4d13-90fd-6860fa933e26 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:54:49.708047 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bc8dd5-60a7-4d13-90fd-6860fa933e26 service swift] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:54:49.716034 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1183/2389] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Jul 26 12:54:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 12:54:55.998836 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72864f-9b3e-461b-af7e-29aa4276c8dc None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:54:55.999447 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72864f-9b3e-461b-af7e-29aa4276c8dc None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:54:55.999447 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72864f-9b3e-461b-af7e-29aa4276c8dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:54:56.020452 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-2a72864f-9b3e-461b-af7e-29aa4276c8dc None None] MFA Rules not processed for user `7a6832c70c0c401985bc4f26a787080f`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:54:56.021653 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-2a72864f-9b3e-461b-af7e-29aa4276c8dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:54:56.079551 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1207/2390] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:54:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:54:59.712876 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=32 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:55:00.074946 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1997e7-a85c-479e-bcdc-83250d490da8 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:55:00.075227 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1997e7-a85c-479e-bcdc-83250d490da8 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:55:00.075499 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1997e7-a85c-479e-bcdc-83250d490da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:55:00.095002 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-0b1997e7-a85c-479e-bcdc-83250d490da8 None None] MFA Rules not processed for user `94efba410653451da2e9977d746633f6`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:55:00.096248 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-0b1997e7-a85c-479e-bcdc-83250d490da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:55:00.103928 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1184/2391] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:55:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:55:06.072118 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=50 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:55:10.089388 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=3 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:55:18.867791 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9874a-5b9d-40b4-9008-7b0d2662ef70 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:55:18.868737 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9874a-5b9d-40b4-9008-7b0d2662ef70 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:55:18.869156 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9874a-5b9d-40b4-9008-7b0d2662ef70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:55:18.902343 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-d9d9874a-5b9d-40b4-9008-7b0d2662ef70 None None] MFA Rules not processed for user `30fc532f464843a3ad35e4d3c387bb3a`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:55:18.903612 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-d9d9874a-5b9d-40b4-9008-7b0d2662ef70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:55:18.922597 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1208/2392] 10.222.0.20 () {58 vars in 967 bytes} [Tue Jul 26 12:55:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:55:18.954992 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1997e7-a85c-479e-bcdc-83250d490da8 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:55:18.967861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1997e7-a85c-479e-bcdc-83250d490da8 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:55:18.973963 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5fc49d-2475-4b16-a91d-de0b10f39ca3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd2483f74d7d48b1a7b8ffe32f63abde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:55:18.974578 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5fc49d-2475-4b16-a91d-de0b10f39ca3 service swift] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:55:18.975045 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5fc49d-2475-4b16-a91d-de0b10f39ca3 service swift] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:55:18.975541 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5fc49d-2475-4b16-a91d-de0b10f39ca3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:55:18.989898 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5fc49d-2475-4b16-a91d-de0b10f39ca3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:55:18.992149 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5fc49d-2475-4b16-a91d-de0b10f39ca3 service swift] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:55:19.009674 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1185/2393] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Jul 26 12:55:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 12:55:21.997784 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b13ee34-fb18-4338-ac27-b9f3042fb2fd None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:55:21.998334 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b13ee34-fb18-4338-ac27-b9f3042fb2fd None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:55:21.998564 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b13ee34-fb18-4338-ac27-b9f3042fb2fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:55:22.025830 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-8b13ee34-fb18-4338-ac27-b9f3042fb2fd None None] MFA Rules not processed for user `1401efb776ab424e8ff4f09eac33d92f`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:55:22.027040 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-8b13ee34-fb18-4338-ac27-b9f3042fb2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:55:22.086470 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1209/2394] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:55:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:55:24.288371 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5fc49d-2475-4b16-a91d-de0b10f39ca3 service swift] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:55:24.346636 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5fc49d-2475-4b16-a91d-de0b10f39ca3 service swift] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:55:24.350254 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5596c3b8-2f22-493e-a80c-b5a2f02ee01c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:55:24.350863 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596c3b8-2f22-493e-a80c-b5a2f02ee01c service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:55:24.351144 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596c3b8-2f22-493e-a80c-b5a2f02ee01c service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:55:24.351415 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596c3b8-2f22-493e-a80c-b5a2f02ee01c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:55:24.360463 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5596c3b8-2f22-493e-a80c-b5a2f02ee01c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:55:24.361912 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5596c3b8-2f22-493e-a80c-b5a2f02ee01c service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:55:24.371395 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1186/2395] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:55:24 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:55:27.822508 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b13ee34-fb18-4338-ac27-b9f3042fb2fd None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:55:27.835939 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b13ee34-fb18-4338-ac27-b9f3042fb2fd None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:55:27.848750 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf82fc6-3413-468b-a4bf-3554d4d98899 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:55:27.850700 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf82fc6-3413-468b-a4bf-3554d4d98899 service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:55:27.852528 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf82fc6-3413-468b-a4bf-3554d4d98899 service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:55:27.852873 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf82fc6-3413-468b-a4bf-3554d4d98899 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:55:27.865514 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf82fc6-3413-468b-a4bf-3554d4d98899 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:55:27.873325 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf82fc6-3413-468b-a4bf-3554d4d98899 service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:55:27.899476 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1210/2396] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:55:27 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:55:34.367190 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=32 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:55:37.881742 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=34 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:55:40.898627 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10368b8c-7978-4dea-97e9-1e766622a054 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:55:40.899173 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10368b8c-7978-4dea-97e9-1e766622a054 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:55:40.899173 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10368b8c-7978-4dea-97e9-1e766622a054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:55:40.919609 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-10368b8c-7978-4dea-97e9-1e766622a054 None None] MFA Rules not processed for user `2b5894762d4b4cc99521bc224d2bddd7`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:55:40.920875 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-10368b8c-7978-4dea-97e9-1e766622a054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:55:40.979510 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1187/2397] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:55:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:55:50.971368 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=25 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:56:12.607021 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf82fc6-3413-468b-a4bf-3554d4d98899 service neutron] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:56:12.608536 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-fbf82fc6-3413-468b-a4bf-3554d4d98899 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:56:12.679233 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf82fc6-3413-468b-a4bf-3554d4d98899 service neutron] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:56:12.683789 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5e2e5-0871-4f02-8ca6-7beb4409262f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:56:12.684424 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5e2e5-0871-4f02-8ca6-7beb4409262f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:56:12.684705 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5e2e5-0871-4f02-8ca6-7beb4409262f admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:56:12.684979 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5e2e5-0871-4f02-8ca6-7beb4409262f admin admin] PATH_INFO: `/v3/users/7a6832c70c0c401985bc4f26a787080f` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:56:12.686670 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe5e2e5-0871-4f02-8ca6-7beb4409262f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a6832c70c0c401985bc4f26a787080f)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:56:12.688911 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe5e2e5-0871-4f02-8ca6-7beb4409262f admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:56:12.735355 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-7fe5e2e5-0871-4f02-8ca6-7beb4409262f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a6832c70c0c401985bc4f26a787080f'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:56:12.742751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-7fe5e2e5-0871-4f02-8ca6-7beb4409262f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a6832c70c0c401985bc4f26a787080f'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:56:12.743387 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-7fe5e2e5-0871-4f02-8ca6-7beb4409262f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a6832c70c0c401985bc4f26a787080f'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:56:12.750159 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-7fe5e2e5-0871-4f02-8ca6-7beb4409262f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a6832c70c0c401985bc4f26a787080f'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:56:12.761927 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-7fe5e2e5-0871-4f02-8ca6-7beb4409262f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a6832c70c0c401985bc4f26a787080f'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:56:12.767717 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1211/2398] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:56:12 2022] DELETE /identity/v3/users/7a6832c70c0c401985bc4f26a787080f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:56:12.948443 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10368b8c-7978-4dea-97e9-1e766622a054 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:56:12.959416 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10368b8c-7978-4dea-97e9-1e766622a054 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:56:13.022413 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62369cb-81c2-4cb9-966c-5fc3a734bbbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:56:13.023272 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62369cb-81c2-4cb9-966c-5fc3a734bbbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:56:13.023552 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62369cb-81c2-4cb9-966c-5fc3a734bbbf admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:56:13.023869 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62369cb-81c2-4cb9-966c-5fc3a734bbbf admin admin] PATH_INFO: `/v3/projects/fa4a3dadafc44483ad16923db53db457` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:56:13.025249 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62369cb-81c2-4cb9-966c-5fc3a734bbbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa4a3dadafc44483ad16923db53db457)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:56:13.027765 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62369cb-81c2-4cb9-966c-5fc3a734bbbf admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:56:13.073321 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-b62369cb-81c2-4cb9-966c-5fc3a734bbbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa4a3dadafc44483ad16923db53db457', 'request_id': 'req-b62369cb-81c2-4cb9-966c-5fc3a734bbbf'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:56:13.079801 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-b62369cb-81c2-4cb9-966c-5fc3a734bbbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa4a3dadafc44483ad16923db53db457', 'request_id': 'req-b62369cb-81c2-4cb9-966c-5fc3a734bbbf'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:56:13.087907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-b62369cb-81c2-4cb9-966c-5fc3a734bbbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa4a3dadafc44483ad16923db53db457', 'request_id': 'req-b62369cb-81c2-4cb9-966c-5fc3a734bbbf'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:56:13.098155 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-b62369cb-81c2-4cb9-966c-5fc3a734bbbf admin admin] The token cache is being invalidate because project fa4a3dadafc44483ad16923db53db457 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:56:13.098445 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-b62369cb-81c2-4cb9-966c-5fc3a734bbbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:56:13.099068 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-b62369cb-81c2-4cb9-966c-5fc3a734bbbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:56:13.101109 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1188/2399] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:56:12 2022] DELETE /identity/v3/projects/fa4a3dadafc44483ad16923db53db457 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:56:14.416995 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5e2e5-0871-4f02-8ca6-7beb4409262f admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:56:14.418535 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-7fe5e2e5-0871-4f02-8ca6-7beb4409262f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:56:14.481663 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5e2e5-0871-4f02-8ca6-7beb4409262f admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:56:14.520687 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:56:14.541047 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb17f7-d41c-4595-8f9a-b655230688e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:56:14.541702 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb17f7-d41c-4595-8f9a-b655230688e6 service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:56:14.541992 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb17f7-d41c-4595-8f9a-b655230688e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:56:14.542256 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb17f7-d41c-4595-8f9a-b655230688e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:56:14.544128 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-dfbb17f7-d41c-4595-8f9a-b655230688e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:56:14.616124 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbb17f7-d41c-4595-8f9a-b655230688e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:56:14.617487 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbb17f7-d41c-4595-8f9a-b655230688e6 service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:56:14.628909 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1212/2400] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:56:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:56:17.652631 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55f946-88e5-43ac-a567-5f687b6c9452 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:56:17.653130 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55f946-88e5-43ac-a567-5f687b6c9452 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:56:17.653245 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55f946-88e5-43ac-a567-5f687b6c9452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:56:17.672703 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-1b55f946-88e5-43ac-a567-5f687b6c9452 None None] MFA Rules not processed for user `17b7e2c1b1b04366b99e9ab892aeb835`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:56:17.673962 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-1b55f946-88e5-43ac-a567-5f687b6c9452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:56:17.730089 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1189/2401] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:56:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:56:17.768041 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb17f7-d41c-4595-8f9a-b655230688e6 service neutron] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:56:17.769443 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-dfbb17f7-d41c-4595-8f9a-b655230688e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:56:17.825358 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb17f7-d41c-4595-8f9a-b655230688e6 service neutron] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:56:17.829169 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c20f28-8825-4dce-9b57-6d6fc6365069 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2b5853838dc40db89c46038c0606107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:56:17.829912 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c20f28-8825-4dce-9b57-6d6fc6365069 service glance] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:56:17.830194 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c20f28-8825-4dce-9b57-6d6fc6365069 service glance] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:56:17.830467 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c20f28-8825-4dce-9b57-6d6fc6365069 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:56:17.832333 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-70c20f28-8825-4dce-9b57-6d6fc6365069 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:56:17.847772 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c20f28-8825-4dce-9b57-6d6fc6365069 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:56:17.849220 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c20f28-8825-4dce-9b57-6d6fc6365069 service glance] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:56:17.864372 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1213/2402] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:56:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:56:27.722739 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=76,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:56:27.857886 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=64 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:56:30.517003 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55f946-88e5-43ac-a567-5f687b6c9452 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:56:30.518473 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-1b55f946-88e5-43ac-a567-5f687b6c9452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:56:30.579339 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55f946-88e5-43ac-a567-5f687b6c9452 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:56:30.582932 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55757486-6f6f-48ea-962d-8d109fd59e56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:56:30.584051 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55757486-6f6f-48ea-962d-8d109fd59e56 service cinder] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:56:30.584342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55757486-6f6f-48ea-962d-8d109fd59e56 service cinder] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:56:30.584614 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55757486-6f6f-48ea-962d-8d109fd59e56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:56:30.586370 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-55757486-6f6f-48ea-962d-8d109fd59e56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:56:30.645482 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55757486-6f6f-48ea-962d-8d109fd59e56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:56:30.647457 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55757486-6f6f-48ea-962d-8d109fd59e56 service cinder] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:56:30.656537 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1190/2403] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:56:30 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:56:33.155692 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c20f28-8825-4dce-9b57-6d6fc6365069 service glance] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:56:33.157089 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-70c20f28-8825-4dce-9b57-6d6fc6365069 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:56:33.171308 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c20f28-8825-4dce-9b57-6d6fc6365069 service glance] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:56:33.176790 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1592bfc2-8a5e-493c-9069-2b114ac0a2e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:56:33.177893 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592bfc2-8a5e-493c-9069-2b114ac0a2e7 service nova] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:56:33.178171 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592bfc2-8a5e-493c-9069-2b114ac0a2e7 service nova] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:56:33.178436 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592bfc2-8a5e-493c-9069-2b114ac0a2e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:56:33.180348 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-1592bfc2-8a5e-493c-9069-2b114ac0a2e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:56:33.238820 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1592bfc2-8a5e-493c-9069-2b114ac0a2e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:56:33.241042 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1592bfc2-8a5e-493c-9069-2b114ac0a2e7 service nova] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:56:33.250344 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1214/2404] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:56:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:56:40.652439 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=48 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:56:43.246055 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=28 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:56:44.507242 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55757486-6f6f-48ea-962d-8d109fd59e56 service cinder] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:56:44.509599 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-55757486-6f6f-48ea-962d-8d109fd59e56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:56:44.528253 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55757486-6f6f-48ea-962d-8d109fd59e56 service cinder] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:56:44.533995 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1ea734-f03d-4c93-916e-f987a32c1569 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:56:44.534775 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1ea734-f03d-4c93-916e-f987a32c1569 service nova] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:56:44.535085 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1ea734-f03d-4c93-916e-f987a32c1569 service nova] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:56:44.535388 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1ea734-f03d-4c93-916e-f987a32c1569 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:56:44.537814 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-aa1ea734-f03d-4c93-916e-f987a32c1569 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:56:44.555011 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1ea734-f03d-4c93-916e-f987a32c1569 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:56:44.556737 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1ea734-f03d-4c93-916e-f987a32c1569 service nova] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:56:44.587131 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1191/2405] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:56:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:56:51.615618 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1592bfc2-8a5e-493c-9069-2b114ac0a2e7 service nova] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:56:51.626775 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1592bfc2-8a5e-493c-9069-2b114ac0a2e7 service nova] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:56:51.636627 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a0611-2599-4ac6-9419-4650d0c35c54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:56:51.637305 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a0611-2599-4ac6-9419-4650d0c35c54 service nova] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:56:51.637666 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a0611-2599-4ac6-9419-4650d0c35c54 service nova] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:56:51.637942 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a0611-2599-4ac6-9419-4650d0c35c54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:56:51.639819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-236a0611-2599-4ac6-9419-4650d0c35c54 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:56:51.653393 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236a0611-2599-4ac6-9419-4650d0c35c54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:56:51.655041 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236a0611-2599-4ac6-9419-4650d0c35c54 service nova] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:56:51.667577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1215/2406] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:56:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:56:51.836523 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1ea734-f03d-4c93-916e-f987a32c1569 service nova] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:56:51.837905 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-aa1ea734-f03d-4c93-916e-f987a32c1569 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:56:51.852006 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1ea734-f03d-4c93-916e-f987a32c1569 service nova] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:56:51.855688 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f6518-b07b-4e41-b309-c5240041aed7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:56:51.856551 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f6518-b07b-4e41-b309-c5240041aed7 service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:56:51.856831 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f6518-b07b-4e41-b309-c5240041aed7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:56:51.857103 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f6518-b07b-4e41-b309-c5240041aed7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:56:51.859232 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-870f6518-b07b-4e41-b309-c5240041aed7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:56:51.930628 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870f6518-b07b-4e41-b309-c5240041aed7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:56:51.932282 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870f6518-b07b-4e41-b309-c5240041aed7 service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:56:51.944029 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1192/2407] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:56:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:56:55.430772 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a0611-2599-4ac6-9419-4650d0c35c54 service nova] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:56:55.432577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-236a0611-2599-4ac6-9419-4650d0c35c54 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:56:55.513331 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f6518-b07b-4e41-b309-c5240041aed7 service neutron] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:56:55.514883 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-870f6518-b07b-4e41-b309-c5240041aed7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:56:55.529899 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f6518-b07b-4e41-b309-c5240041aed7 service neutron] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:56:55.539993 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43521b74-6706-4f5d-b5c8-fff5f0f5f0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:56:55.540649 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43521b74-6706-4f5d-b5c8-fff5f0f5f0cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:56:55.540957 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43521b74-6706-4f5d-b5c8-fff5f0f5f0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:56:55.541412 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43521b74-6706-4f5d-b5c8-fff5f0f5f0cd admin admin] PATH_INFO: `/v3/users/2b5894762d4b4cc99521bc224d2bddd7` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:56:55.542665 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43521b74-6706-4f5d-b5c8-fff5f0f5f0cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b5894762d4b4cc99521bc224d2bddd7)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:56:55.545025 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43521b74-6706-4f5d-b5c8-fff5f0f5f0cd admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:56:55.554067 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a0611-2599-4ac6-9419-4650d0c35c54 service nova] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:56:55.559812 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b1e4c1-ab86-406b-aa61-c8107773f8ef service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec49a03aed774c439459883bd5442d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:56:55.560443 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1e4c1-ab86-406b-aa61-c8107773f8ef service placement] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:56:55.560733 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1e4c1-ab86-406b-aa61-c8107773f8ef service placement] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:56:55.561007 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1e4c1-ab86-406b-aa61-c8107773f8ef service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:56:55.562791 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-40b1e4c1-ab86-406b-aa61-c8107773f8ef service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:56:55.577959 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b1e4c1-ab86-406b-aa61-c8107773f8ef service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:56:55.580057 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b1e4c1-ab86-406b-aa61-c8107773f8ef service placement] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:56:55.593369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1216/2408] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:56:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:56:55.602957 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-43521b74-6706-4f5d-b5c8-fff5f0f5f0cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b5894762d4b4cc99521bc224d2bddd7'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:56:55.603930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-43521b74-6706-4f5d-b5c8-fff5f0f5f0cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b5894762d4b4cc99521bc224d2bddd7'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:56:55.604560 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-43521b74-6706-4f5d-b5c8-fff5f0f5f0cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b5894762d4b4cc99521bc224d2bddd7'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:56:55.618600 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-43521b74-6706-4f5d-b5c8-fff5f0f5f0cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b5894762d4b4cc99521bc224d2bddd7'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:56:55.626233 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-43521b74-6706-4f5d-b5c8-fff5f0f5f0cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b5894762d4b4cc99521bc224d2bddd7'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:56:55.642668 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1193/2409] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:56:55 2022] DELETE /identity/v3/users/2b5894762d4b4cc99521bc224d2bddd7 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:56:55.821558 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d696d80c-d107-4601-8be2-86e7cbdb56b8 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:56:55.821807 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d696d80c-d107-4601-8be2-86e7cbdb56b8 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:56:55.822235 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d696d80c-d107-4601-8be2-86e7cbdb56b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:56:55.834582 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43521b74-6706-4f5d-b5c8-fff5f0f5f0cd admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:56:55.836145 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-43521b74-6706-4f5d-b5c8-fff5f0f5f0cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:56:55.853191 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-d696d80c-d107-4601-8be2-86e7cbdb56b8 None None] MFA Rules not processed for user `1401efb776ab424e8ff4f09eac33d92f`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:56:55.855345 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-d696d80c-d107-4601-8be2-86e7cbdb56b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:56:55.896657 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:56:55.912620 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43521b74-6706-4f5d-b5c8-fff5f0f5f0cd admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:56:55.920843 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87b28d7-8e6f-46ee-979d-2e738e619efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:56:55.923168 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b28d7-8e6f-46ee-979d-2e738e619efa admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:56:55.923631 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b28d7-8e6f-46ee-979d-2e738e619efa admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:56:55.924060 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b28d7-8e6f-46ee-979d-2e738e619efa admin admin] PATH_INFO: `/v3/projects/d92bfe73c0a54823a88a89f9ae627134` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:56:55.925237 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87b28d7-8e6f-46ee-979d-2e738e619efa admin admin] RBAC: Authorizing `identity:delete_project(project_id=d92bfe73c0a54823a88a89f9ae627134)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:56:55.927337 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87b28d7-8e6f-46ee-979d-2e738e619efa admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:56:55.928096 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1217/2410] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:56:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:56:55.983819 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-b87b28d7-8e6f-46ee-979d-2e738e619efa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd92bfe73c0a54823a88a89f9ae627134', 'request_id': 'req-b87b28d7-8e6f-46ee-979d-2e738e619efa'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:56:55.990326 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-b87b28d7-8e6f-46ee-979d-2e738e619efa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd92bfe73c0a54823a88a89f9ae627134', 'request_id': 'req-b87b28d7-8e6f-46ee-979d-2e738e619efa'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:56:55.997800 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-b87b28d7-8e6f-46ee-979d-2e738e619efa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd92bfe73c0a54823a88a89f9ae627134', 'request_id': 'req-b87b28d7-8e6f-46ee-979d-2e738e619efa'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:56:56.009251 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-b87b28d7-8e6f-46ee-979d-2e738e619efa admin admin] The token cache is being invalidate because project d92bfe73c0a54823a88a89f9ae627134 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:56:56.009545 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-b87b28d7-8e6f-46ee-979d-2e738e619efa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:56:56.010199 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-b87b28d7-8e6f-46ee-979d-2e738e619efa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:56:56.011911 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1194/2411] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:56:55 2022] DELETE /identity/v3/projects/d92bfe73c0a54823a88a89f9ae627134 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:56:56.016977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473465e1-3e3d-4b35-bdc0-c3368811b5e0 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:56:56.017264 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473465e1-3e3d-4b35-bdc0-c3368811b5e0 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:56:56.017539 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473465e1-3e3d-4b35-bdc0-c3368811b5e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:56:56.039521 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-473465e1-3e3d-4b35-bdc0-c3368811b5e0 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:56:56.040892 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-473465e1-3e3d-4b35-bdc0-c3368811b5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:56:56.111190 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1218/2412] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:56:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:56:56.119387 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8341c8-5204-4fa8-bc95-7af7987a7a59 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:56:56.119866 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8341c8-5204-4fa8-bc95-7af7987a7a59 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:56:56.120220 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8341c8-5204-4fa8-bc95-7af7987a7a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:56:56.141178 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-bc8341c8-5204-4fa8-bc95-7af7987a7a59 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:56:56.142500 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-bc8341c8-5204-4fa8-bc95-7af7987a7a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:56:56.150679 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1195/2413] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:56:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:56:56.154960 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473465e1-3e3d-4b35-bdc0-c3368811b5e0 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:56:56.168747 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473465e1-3e3d-4b35-bdc0-c3368811b5e0 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:56:56.173519 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7428d4f6-8eaa-46ad-a2e8-7614c399a70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:56:56.174136 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7428d4f6-8eaa-46ad-a2e8-7614c399a70c admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:56:56.174419 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7428d4f6-8eaa-46ad-a2e8-7614c399a70c admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:56:56.174727 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7428d4f6-8eaa-46ad-a2e8-7614c399a70c admin admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:56:56.175435 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7428d4f6-8eaa-46ad-a2e8-7614c399a70c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:56:56.175771 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7428d4f6-8eaa-46ad-a2e8-7614c399a70c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:56:56.178025 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7428d4f6-8eaa-46ad-a2e8-7614c399a70c admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:56:56.190466 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1219/2414] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:56:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:56:56.195456 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8341c8-5204-4fa8-bc95-7af7987a7a59 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:56:56.204823 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8341c8-5204-4fa8-bc95-7af7987a7a59 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:56:56.210033 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42805710-732e-4f16-9f8e-059b35f454ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:56:56.210648 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42805710-732e-4f16-9f8e-059b35f454ec admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:56:56.210931 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42805710-732e-4f16-9f8e-059b35f454ec admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:56:56.211204 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42805710-732e-4f16-9f8e-059b35f454ec admin admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:56:56.211910 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42805710-732e-4f16-9f8e-059b35f454ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:56:56.213850 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42805710-732e-4f16-9f8e-059b35f454ec admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:56:56.242605 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1196/2415] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:56:56 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:56:56.246758 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7428d4f6-8eaa-46ad-a2e8-7614c399a70c admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:56:56.257656 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7428d4f6-8eaa-46ad-a2e8-7614c399a70c admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:56:56.262287 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:56:56.265360 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42805710-732e-4f16-9f8e-059b35f454ec admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:56:56.267110 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-42805710-732e-4f16-9f8e-059b35f454ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:56:56.315162 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948f62d-0d62-4ebc-851b-f7a1fc1bada0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:56:56.315811 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948f62d-0d62-4ebc-851b-f7a1fc1bada0 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:56:56.316097 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948f62d-0d62-4ebc-851b-f7a1fc1bada0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:56:56.316370 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948f62d-0d62-4ebc-851b-f7a1fc1bada0 admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:56:56.317031 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9948f62d-0d62-4ebc-851b-f7a1fc1bada0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:56:56.319528 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9948f62d-0d62-4ebc-851b-f7a1fc1bada0 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:56:56.324815 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42805710-732e-4f16-9f8e-059b35f454ec admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:56:56.329667 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf722cfa-bf99-4705-ab91-56ceb7b2f90f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:56:56.330450 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf722cfa-bf99-4705-ab91-56ceb7b2f90f service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:56:56.330745 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf722cfa-bf99-4705-ab91-56ceb7b2f90f service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:56:56.331003 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf722cfa-bf99-4705-ab91-56ceb7b2f90f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:56:56.332903 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-cf722cfa-bf99-4705-ab91-56ceb7b2f90f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:56:56.344220 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1220/2416] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:56:56 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:56:56.344404 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf722cfa-bf99-4705-ab91-56ceb7b2f90f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:56:56.344694 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf722cfa-bf99-4705-ab91-56ceb7b2f90f service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:56:56.345276 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948f62d-0d62-4ebc-851b-f7a1fc1bada0 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:56:56.353482 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1197/2417] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:56:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:56:56.357610 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948f62d-0d62-4ebc-851b-f7a1fc1bada0 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:56:56.362620 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26323f32-6c7d-4734-969c-d19d50b7ed03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:56:56.363293 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26323f32-6c7d-4734-969c-d19d50b7ed03 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:56:56.363579 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26323f32-6c7d-4734-969c-d19d50b7ed03 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:56:56.363901 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26323f32-6c7d-4734-969c-d19d50b7ed03 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:56:56.365135 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26323f32-6c7d-4734-969c-d19d50b7ed03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:56:56.366742 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26323f32-6c7d-4734-969c-d19d50b7ed03 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:56:56.377809 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1221/2418] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:56:56.382949 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf722cfa-bf99-4705-ab91-56ceb7b2f90f service neutron] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:56:56.394545 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf722cfa-bf99-4705-ab91-56ceb7b2f90f service neutron] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:56:56.399260 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b1f20-f53b-47b5-97cb-5671fd01a207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:56:56.399908 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b1f20-f53b-47b5-97cb-5671fd01a207 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:56:56.400190 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b1f20-f53b-47b5-97cb-5671fd01a207 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:56:56.400465 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b1f20-f53b-47b5-97cb-5671fd01a207 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:56:56.401170 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843b1f20-f53b-47b5-97cb-5671fd01a207 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:56:56.402774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843b1f20-f53b-47b5-97cb-5671fd01a207 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:56:56.412977 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1198/2419] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:56:56.417278 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26323f32-6c7d-4734-969c-d19d50b7ed03 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:56:56.427829 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26323f32-6c7d-4734-969c-d19d50b7ed03 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:56:56.432612 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21565c14-4fbc-4814-a62c-686cd293479b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:56:56.433292 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21565c14-4fbc-4814-a62c-686cd293479b admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:56:56.433575 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21565c14-4fbc-4814-a62c-686cd293479b admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:56:56.433873 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21565c14-4fbc-4814-a62c-686cd293479b admin admin] PATH_INFO: `/v3/projects/429715c1b71244bcbbf9f30190d25823/users/d480ea8c18634b88b3248a80422aa798/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:56:56.461152 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21565c14-4fbc-4814-a62c-686cd293479b admin admin] RBAC: Authorizing `identity:create_grant(project_id=429715c1b71244bcbbf9f30190d25823, user_id=d480ea8c18634b88b3248a80422aa798, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:56:56.463886 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21565c14-4fbc-4814-a62c-686cd293479b admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:56:56.477743 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1222/2420] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:56:56 2022] PUT /identity/v3/projects/429715c1b71244bcbbf9f30190d25823/users/d480ea8c18634b88b3248a80422aa798/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:56:56.482766 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e58d0-0cf5-45a5-8381-5d9c387c1c4b None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:56:56.483059 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e58d0-0cf5-45a5-8381-5d9c387c1c4b None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:56:56.483340 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e58d0-0cf5-45a5-8381-5d9c387c1c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:56:56.503498 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-3b3e58d0-0cf5-45a5-8381-5d9c387c1c4b None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:56:56.504992 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-3b3e58d0-0cf5-45a5-8381-5d9c387c1c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:56:56.549320 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:56:56.578615 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1199/2421] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:56:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:56:56.589680 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21565c14-4fbc-4814-a62c-686cd293479b admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:56:56.591438 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-21565c14-4fbc-4814-a62c-686cd293479b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:56:56.655442 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21565c14-4fbc-4814-a62c-686cd293479b admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:56:56.659424 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc636dc-3549-433e-b513-d31f842f8235 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:56:56.660092 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc636dc-3549-433e-b513-d31f842f8235 service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:56:56.660371 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc636dc-3549-433e-b513-d31f842f8235 service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:56:56.660645 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc636dc-3549-433e-b513-d31f842f8235 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:56:56.672112 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc636dc-3549-433e-b513-d31f842f8235 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:56:56.674178 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc636dc-3549-433e-b513-d31f842f8235 service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:56:56.682810 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3e58d0-0cf5-45a5-8381-5d9c387c1c4b None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:56:56.684373 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-3b3e58d0-0cf5-45a5-8381-5d9c387c1c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:56:56.685844 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1223/2422] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:56:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:56:56.761435 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3e58d0-0cf5-45a5-8381-5d9c387c1c4b None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:56:56.767088 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8109c6-1f25-4637-9c0f-1a4d3cdd6239 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec49a03aed774c439459883bd5442d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:56:56.767756 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8109c6-1f25-4637-9c0f-1a4d3cdd6239 service placement] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:56:56.768023 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8109c6-1f25-4637-9c0f-1a4d3cdd6239 service placement] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:56:56.768303 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8109c6-1f25-4637-9c0f-1a4d3cdd6239 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:56:56.770160 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-6a8109c6-1f25-4637-9c0f-1a4d3cdd6239 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:56:56.783368 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8109c6-1f25-4637-9c0f-1a4d3cdd6239 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:56:56.785038 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8109c6-1f25-4637-9c0f-1a4d3cdd6239 service placement] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:56:56.796858 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1200/2423] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:56:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 12:56:58.651546 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc636dc-3549-433e-b513-d31f842f8235 service neutron] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:56:58.661359 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc636dc-3549-433e-b513-d31f842f8235 service neutron] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:56:58.665164 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaef31d5-7c07-4ebe-885a-937f8912a94d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:56:58.666015 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef31d5-7c07-4ebe-885a-937f8912a94d service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:56:58.666290 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef31d5-7c07-4ebe-885a-937f8912a94d service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:56:58.666556 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef31d5-7c07-4ebe-885a-937f8912a94d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:56:58.669218 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-aaef31d5-7c07-4ebe-885a-937f8912a94d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:56:58.681647 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaef31d5-7c07-4ebe-885a-937f8912a94d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:56:58.683254 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaef31d5-7c07-4ebe-885a-937f8912a94d service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:56:58.693123 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1224/2424] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:56:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:57:01.290895 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d172e025-49c1-4030-8828-13daf5177547 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:57:01.291314 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d172e025-49c1-4030-8828-13daf5177547 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:57:01.291459 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d172e025-49c1-4030-8828-13daf5177547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:57:01.338647 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-d172e025-49c1-4030-8828-13daf5177547 None None] MFA Rules not processed for user `d480ea8c18634b88b3248a80422aa798`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:57:01.341001 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-d172e025-49c1-4030-8828-13daf5177547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:57:01.413531 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1201/2425] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:57:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:57:01.428583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaef31d5-7c07-4ebe-885a-937f8912a94d service neutron] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:57:01.430082 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-aaef31d5-7c07-4ebe-885a-937f8912a94d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:57:01.484959 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:57:01.510041 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaef31d5-7c07-4ebe-885a-937f8912a94d service neutron] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:57:01.515564 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b39e49-7096-4f69-aa84-178540e26adf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:57:01.516330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b39e49-7096-4f69-aa84-178540e26adf service nova] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:57:01.516617 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b39e49-7096-4f69-aa84-178540e26adf service nova] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:57:01.516895 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b39e49-7096-4f69-aa84-178540e26adf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:57:01.525590 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b39e49-7096-4f69-aa84-178540e26adf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:57:01.527251 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b39e49-7096-4f69-aa84-178540e26adf service nova] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:57:01.537132 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1225/2426] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:57:01 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:57:02.961229 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfc8db-7770-415e-b986-977a5a8dec6b None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:57:02.961793 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfc8db-7770-415e-b986-977a5a8dec6b None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:57:02.962016 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfc8db-7770-415e-b986-977a5a8dec6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:57:02.984560 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-36cfc8db-7770-415e-b986-977a5a8dec6b None None] MFA Rules not processed for user `d480ea8c18634b88b3248a80422aa798`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:57:02.986246 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-36cfc8db-7770-415e-b986-977a5a8dec6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:57:03.001350 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1202/2427] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:57:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:57:03.320092 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b39e49-7096-4f69-aa84-178540e26adf service nova] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:57:03.341510 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b39e49-7096-4f69-aa84-178540e26adf service nova] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:57:03.351258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7dfede-f76a-4fd8-9ceb-65f9babcd3fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:57:03.352192 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7dfede-f76a-4fd8-9ceb-65f9babcd3fb service nova] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:57:03.352688 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7dfede-f76a-4fd8-9ceb-65f9babcd3fb service nova] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:57:03.353170 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7dfede-f76a-4fd8-9ceb-65f9babcd3fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:57:03.356975 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfc8db-7770-415e-b986-977a5a8dec6b None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:57:03.357828 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-9e7dfede-f76a-4fd8-9ceb-65f9babcd3fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:57:03.368762 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfc8db-7770-415e-b986-977a5a8dec6b None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:57:03.374019 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad85cb7-b597-4ae7-bd8e-cf05463035b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:57:03.375054 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad85cb7-b597-4ae7-bd8e-cf05463035b9 service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:57:03.375559 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad85cb7-b597-4ae7-bd8e-cf05463035b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:57:03.376078 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad85cb7-b597-4ae7-bd8e-cf05463035b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:57:03.380594 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-0ad85cb7-b597-4ae7-bd8e-cf05463035b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:57:03.381426 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7dfede-f76a-4fd8-9ceb-65f9babcd3fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:57:03.383047 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7dfede-f76a-4fd8-9ceb-65f9babcd3fb service nova] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:57:03.399318 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1226/2428] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:57:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:57:03.400064 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad85cb7-b597-4ae7-bd8e-cf05463035b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:57:03.401954 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad85cb7-b597-4ae7-bd8e-cf05463035b9 service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:57:03.414584 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1203/2429] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:57:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 12:57:04.934637 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7dfede-f76a-4fd8-9ceb-65f9babcd3fb service nova] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:57:04.937072 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-9e7dfede-f76a-4fd8-9ceb-65f9babcd3fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:57:04.955350 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7dfede-f76a-4fd8-9ceb-65f9babcd3fb service nova] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:57:04.963186 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60531c57-0902-40ea-aa37-9e050693143c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:57:04.964217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60531c57-0902-40ea-aa37-9e050693143c service nova] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:57:04.964750 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60531c57-0902-40ea-aa37-9e050693143c service nova] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:57:04.965179 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60531c57-0902-40ea-aa37-9e050693143c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:57:04.967144 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-60531c57-0902-40ea-aa37-9e050693143c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:57:05.045297 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60531c57-0902-40ea-aa37-9e050693143c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:57:05.047575 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60531c57-0902-40ea-aa37-9e050693143c service nova] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:57:05.058848 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1227/2430] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:57:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:57:13.409975 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=79 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:57:15.053701 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=44 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:57:26.243066 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f1e21-854a-43bb-8775-e5fe7860ff0f None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:57:26.243838 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f1e21-854a-43bb-8775-e5fe7860ff0f None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:57:26.243838 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f1e21-854a-43bb-8775-e5fe7860ff0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:57:26.262990 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-585f1e21-854a-43bb-8775-e5fe7860ff0f None None] MFA Rules not processed for user `30fc532f464843a3ad35e4d3c387bb3a`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:57:26.264183 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-585f1e21-854a-43bb-8775-e5fe7860ff0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:57:26.319156 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1204/2431] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Jul 26 12:57:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:57:26.365476 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60531c57-0902-40ea-aa37-9e050693143c service nova] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:57:26.366941 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-60531c57-0902-40ea-aa37-9e050693143c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:57:26.436117 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60531c57-0902-40ea-aa37-9e050693143c service nova] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:57:26.441075 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42904705-e66b-4251-b9fa-7754e0566658 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd2483f74d7d48b1a7b8ffe32f63abde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:57:26.441679 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42904705-e66b-4251-b9fa-7754e0566658 service swift] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:57:26.441960 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42904705-e66b-4251-b9fa-7754e0566658 service swift] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:57:26.442231 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42904705-e66b-4251-b9fa-7754e0566658 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:57:26.450957 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42904705-e66b-4251-b9fa-7754e0566658 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:57:26.452560 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42904705-e66b-4251-b9fa-7754e0566658 service swift] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:57:26.460509 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1228/2432] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Jul 26 12:57:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 12:57:36.312040 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=25 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:57:36.457309 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=32 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:57:42.133291 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94bdba4-2e46-4bd8-99e2-9f158a7a65f1 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:57:42.133759 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94bdba4-2e46-4bd8-99e2-9f158a7a65f1 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:57:42.133859 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94bdba4-2e46-4bd8-99e2-9f158a7a65f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:57:42.154311 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-a94bdba4-2e46-4bd8-99e2-9f158a7a65f1 None None] MFA Rules not processed for user `94efba410653451da2e9977d746633f6`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:57:42.155514 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-a94bdba4-2e46-4bd8-99e2-9f158a7a65f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:57:42.211816 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1205/2433] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:57:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:57:43.021478 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85b366-3e1d-4b8a-8a34-97c9e212a9ff None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:57:43.022023 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85b366-3e1d-4b8a-8a34-97c9e212a9ff None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:57:43.022023 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85b366-3e1d-4b8a-8a34-97c9e212a9ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:57:43.044348 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-ae85b366-3e1d-4b8a-8a34-97c9e212a9ff None None] MFA Rules not processed for user `30fc532f464843a3ad35e4d3c387bb3a`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:57:43.045576 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-ae85b366-3e1d-4b8a-8a34-97c9e212a9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:57:43.055248 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1229/2434] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Jul 26 12:57:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:57:43.068931 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94bdba4-2e46-4bd8-99e2-9f158a7a65f1 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:57:43.083845 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94bdba4-2e46-4bd8-99e2-9f158a7a65f1 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:57:43.090117 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164aadad-f0a0-4f7b-a374-536ec6a40e85 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd2483f74d7d48b1a7b8ffe32f63abde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:57:43.090829 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164aadad-f0a0-4f7b-a374-536ec6a40e85 service swift] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:57:43.091348 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164aadad-f0a0-4f7b-a374-536ec6a40e85 service swift] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:57:43.091626 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164aadad-f0a0-4f7b-a374-536ec6a40e85 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:57:43.101781 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164aadad-f0a0-4f7b-a374-536ec6a40e85 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:57:43.103768 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164aadad-f0a0-4f7b-a374-536ec6a40e85 service swift] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:57:43.131673 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1206/2435] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Jul 26 12:57:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 12:57:53.037490 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=3 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:57:53.110323 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=31 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:57:59.364897 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f784b0-c1f0-45ef-acd5-bb05fce44d88 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:57:59.365360 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f784b0-c1f0-45ef-acd5-bb05fce44d88 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:57:59.365521 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f784b0-c1f0-45ef-acd5-bb05fce44d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:57:59.385616 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-91f784b0-c1f0-45ef-acd5-bb05fce44d88 None None] MFA Rules not processed for user `30fc532f464843a3ad35e4d3c387bb3a`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:57:59.387087 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-91f784b0-c1f0-45ef-acd5-bb05fce44d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:57:59.394827 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1230/2436] 10.222.0.20 () {58 vars in 967 bytes} [Tue Jul 26 12:57:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:57:59.407879 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164aadad-f0a0-4f7b-a374-536ec6a40e85 service swift] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:57:59.418957 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164aadad-f0a0-4f7b-a374-536ec6a40e85 service swift] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:57:59.424473 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7a030a-f109-4326-8f02-9388480596d3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd2483f74d7d48b1a7b8ffe32f63abde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:57:59.425136 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a030a-f109-4326-8f02-9388480596d3 service swift] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:57:59.425423 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a030a-f109-4326-8f02-9388480596d3 service swift] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:57:59.425829 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a030a-f109-4326-8f02-9388480596d3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:57:59.435221 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7a030a-f109-4326-8f02-9388480596d3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:57:59.436850 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7a030a-f109-4326-8f02-9388480596d3 service swift] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:57:59.446033 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1207/2437] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Jul 26 12:57:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 26 12:58:09.379437 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=3 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:58:09.441831 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=6 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:58:19.708269 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f784b0-c1f0-45ef-acd5-bb05fce44d88 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:19.717666 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f784b0-c1f0-45ef-acd5-bb05fce44d88 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:19.721358 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b22a7b6-ebcd-472b-89b9-ea9f41fefe7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:19.721980 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22a7b6-ebcd-472b-89b9-ea9f41fefe7d service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:19.722266 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22a7b6-ebcd-472b-89b9-ea9f41fefe7d service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:19.722544 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22a7b6-ebcd-472b-89b9-ea9f41fefe7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:19.724401 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-2b22a7b6-ebcd-472b-89b9-ea9f41fefe7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:19.736377 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b22a7b6-ebcd-472b-89b9-ea9f41fefe7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:19.737818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b22a7b6-ebcd-472b-89b9-ea9f41fefe7d service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:19.747930 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1231/2438] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:58:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:58:23.727033 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7a030a-f109-4326-8f02-9388480596d3 service swift] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:23.728719 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-ee7a030a-f109-4326-8f02-9388480596d3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:23.744920 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7a030a-f109-4326-8f02-9388480596d3 service swift] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:23.752302 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da404f50-7cdf-4d9b-a297-fad07a75141e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:23.753346 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da404f50-7cdf-4d9b-a297-fad07a75141e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:23.753853 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da404f50-7cdf-4d9b-a297-fad07a75141e admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:23.755131 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da404f50-7cdf-4d9b-a297-fad07a75141e admin admin] PATH_INFO: `/v3/users/1401efb776ab424e8ff4f09eac33d92f` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:23.758043 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da404f50-7cdf-4d9b-a297-fad07a75141e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1401efb776ab424e8ff4f09eac33d92f)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:23.760961 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da404f50-7cdf-4d9b-a297-fad07a75141e admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:23.828400 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-da404f50-7cdf-4d9b-a297-fad07a75141e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1401efb776ab424e8ff4f09eac33d92f'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:23.829257 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-da404f50-7cdf-4d9b-a297-fad07a75141e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1401efb776ab424e8ff4f09eac33d92f'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:23.829988 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-da404f50-7cdf-4d9b-a297-fad07a75141e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1401efb776ab424e8ff4f09eac33d92f'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:23.847237 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-da404f50-7cdf-4d9b-a297-fad07a75141e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1401efb776ab424e8ff4f09eac33d92f'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:23.861890 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-da404f50-7cdf-4d9b-a297-fad07a75141e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1401efb776ab424e8ff4f09eac33d92f'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:23.880448 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1208/2439] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:58:23 2022] DELETE /identity/v3/users/1401efb776ab424e8ff4f09eac33d92f => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:58:24.153920 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b22a7b6-ebcd-472b-89b9-ea9f41fefe7d service neutron] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:24.164775 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b22a7b6-ebcd-472b-89b9-ea9f41fefe7d service neutron] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:24.227249 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcfa981-1b4c-4e1b-8081-55b9f6e63fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:24.227930 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcfa981-1b4c-4e1b-8081-55b9f6e63fb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:24.228216 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcfa981-1b4c-4e1b-8081-55b9f6e63fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:24.228485 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcfa981-1b4c-4e1b-8081-55b9f6e63fb1 admin admin] PATH_INFO: `/v3/projects/01fca015df86428b9356df3f127327c2` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:24.229713 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcfa981-1b4c-4e1b-8081-55b9f6e63fb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01fca015df86428b9356df3f127327c2)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:24.231919 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcfa981-1b4c-4e1b-8081-55b9f6e63fb1 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:24.281764 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-6bcfa981-1b4c-4e1b-8081-55b9f6e63fb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01fca015df86428b9356df3f127327c2', 'request_id': 'req-6bcfa981-1b4c-4e1b-8081-55b9f6e63fb1'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:24.288854 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-6bcfa981-1b4c-4e1b-8081-55b9f6e63fb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01fca015df86428b9356df3f127327c2', 'request_id': 'req-6bcfa981-1b4c-4e1b-8081-55b9f6e63fb1'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:24.297242 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-6bcfa981-1b4c-4e1b-8081-55b9f6e63fb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01fca015df86428b9356df3f127327c2', 'request_id': 'req-6bcfa981-1b4c-4e1b-8081-55b9f6e63fb1'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:24.313521 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-6bcfa981-1b4c-4e1b-8081-55b9f6e63fb1 admin admin] The token cache is being invalidate because project 01fca015df86428b9356df3f127327c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:58:24.313739 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-6bcfa981-1b4c-4e1b-8081-55b9f6e63fb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:24.314397 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-6bcfa981-1b4c-4e1b-8081-55b9f6e63fb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:24.316382 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1232/2440] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:58:24 2022] DELETE /identity/v3/projects/01fca015df86428b9356df3f127327c2 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:58:24.321071 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a70e1a6-271a-482f-b7fe-12948a31770e None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:24.321619 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a70e1a6-271a-482f-b7fe-12948a31770e None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:24.321899 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a70e1a6-271a-482f-b7fe-12948a31770e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:24.343973 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-8a70e1a6-271a-482f-b7fe-12948a31770e None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:58:24.345199 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-8a70e1a6-271a-482f-b7fe-12948a31770e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:24.421061 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1209/2441] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:58:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:58:24.422724 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a1f3c-c15d-4f95-9efc-fe339bbf4919 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:24.423016 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a1f3c-c15d-4f95-9efc-fe339bbf4919 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:24.423291 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a1f3c-c15d-4f95-9efc-fe339bbf4919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:24.444392 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-c13a1f3c-c15d-4f95-9efc-fe339bbf4919 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:58:24.445609 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-c13a1f3c-c15d-4f95-9efc-fe339bbf4919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:24.454176 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1233/2442] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:58:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:58:24.458505 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a70e1a6-271a-482f-b7fe-12948a31770e None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:24.469461 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a70e1a6-271a-482f-b7fe-12948a31770e None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:24.474323 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b865714f-3176-4cc5-bec5-5f096cf0ef16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:24.474940 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865714f-3176-4cc5-bec5-5f096cf0ef16 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:24.475224 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865714f-3176-4cc5-bec5-5f096cf0ef16 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:24.476069 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865714f-3176-4cc5-bec5-5f096cf0ef16 admin admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:24.476996 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b865714f-3176-4cc5-bec5-5f096cf0ef16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:58:24.477313 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b865714f-3176-4cc5-bec5-5f096cf0ef16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:24.478940 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b865714f-3176-4cc5-bec5-5f096cf0ef16 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:24.491300 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1210/2443] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:58:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:58:24.495669 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a1f3c-c15d-4f95-9efc-fe339bbf4919 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:24.504916 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a1f3c-c15d-4f95-9efc-fe339bbf4919 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:24.510730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea77b40-bc35-433b-a824-9c3bd7a109f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:24.511370 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea77b40-bc35-433b-a824-9c3bd7a109f4 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:24.511673 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea77b40-bc35-433b-a824-9c3bd7a109f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:24.511954 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea77b40-bc35-433b-a824-9c3bd7a109f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:24.512658 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea77b40-bc35-433b-a824-9c3bd7a109f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:24.514594 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea77b40-bc35-433b-a824-9c3bd7a109f4 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:24.542845 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1234/2444] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:58:24 2022] POST /identity/v3/projects => generated 355 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:58:24.546341 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b865714f-3176-4cc5-bec5-5f096cf0ef16 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:24.556877 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b865714f-3176-4cc5-bec5-5f096cf0ef16 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:24.613635 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35053c0-b6bb-4efc-b990-36c5b3a3f955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:24.614251 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35053c0-b6bb-4efc-b990-36c5b3a3f955 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:24.614532 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35053c0-b6bb-4efc-b990-36c5b3a3f955 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:24.614805 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35053c0-b6bb-4efc-b990-36c5b3a3f955 admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:24.615473 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35053c0-b6bb-4efc-b990-36c5b3a3f955 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:24.617444 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35053c0-b6bb-4efc-b990-36c5b3a3f955 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:24.625347 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:58:24.641473 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1211/2445] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:58:24 2022] POST /identity/v3/users => generated 278 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:58:24.645633 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea77b40-bc35-433b-a824-9c3bd7a109f4 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:24.656933 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea77b40-bc35-433b-a824-9c3bd7a109f4 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:24.661682 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf46a7-3e14-487a-bb0c-c2a762f01c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:24.662281 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf46a7-3e14-487a-bb0c-c2a762f01c37 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:24.662557 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf46a7-3e14-487a-bb0c-c2a762f01c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:24.662827 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf46a7-3e14-487a-bb0c-c2a762f01c37 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:24.663513 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caf46a7-3e14-487a-bb0c-c2a762f01c37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:24.665166 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caf46a7-3e14-487a-bb0c-c2a762f01c37 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:24.675737 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1235/2446] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:58:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:58:24.679923 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35053c0-b6bb-4efc-b990-36c5b3a3f955 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:24.689915 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35053c0-b6bb-4efc-b990-36c5b3a3f955 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:24.695133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e91398-3897-4f99-a168-03e9d808473d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:24.695774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e91398-3897-4f99-a168-03e9d808473d admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:24.696061 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e91398-3897-4f99-a168-03e9d808473d admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:24.696351 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e91398-3897-4f99-a168-03e9d808473d admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:24.697052 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e91398-3897-4f99-a168-03e9d808473d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:24.698669 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e91398-3897-4f99-a168-03e9d808473d admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:24.711343 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1212/2447] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:58:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:58:24.715599 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf46a7-3e14-487a-bb0c-c2a762f01c37 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:24.726316 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf46a7-3e14-487a-bb0c-c2a762f01c37 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:24.731070 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357808d4-d646-4b9c-bf29-a20dda74f3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:24.731773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357808d4-d646-4b9c-bf29-a20dda74f3e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:24.732060 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357808d4-d646-4b9c-bf29-a20dda74f3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:24.732337 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357808d4-d646-4b9c-bf29-a20dda74f3e4 admin admin] PATH_INFO: `/v3/projects/9bc992a06b094a1daa87cde412e8315e/users/c96ea8a9941540cfa66bde62c66e377c/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:24.748575 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357808d4-d646-4b9c-bf29-a20dda74f3e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bc992a06b094a1daa87cde412e8315e, user_id=c96ea8a9941540cfa66bde62c66e377c, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:24.751628 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357808d4-d646-4b9c-bf29-a20dda74f3e4 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:24.764891 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1236/2448] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:58:24 2022] PUT /identity/v3/projects/9bc992a06b094a1daa87cde412e8315e/users/c96ea8a9941540cfa66bde62c66e377c/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:58:24.776016 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d28a0b-84da-42cf-b1c0-3272a0ef25cc None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:24.776499 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d28a0b-84da-42cf-b1c0-3272a0ef25cc None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:24.776610 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d28a0b-84da-42cf-b1c0-3272a0ef25cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:24.805146 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-15d28a0b-84da-42cf-b1c0-3272a0ef25cc None None] MFA Rules not processed for user `c96ea8a9941540cfa66bde62c66e377c`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:58:24.806370 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-15d28a0b-84da-42cf-b1c0-3272a0ef25cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:24.871836 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1213/2449] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:58:24 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:58:24.877519 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357808d4-d646-4b9c-bf29-a20dda74f3e4 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:24.891029 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:58:24.947461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357808d4-d646-4b9c-bf29-a20dda74f3e4 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:24.952037 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60787383-1ecf-48ce-98a6-87f7a2fdacb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:24.952642 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60787383-1ecf-48ce-98a6-87f7a2fdacb7 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:24.952919 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60787383-1ecf-48ce-98a6-87f7a2fdacb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:24.953188 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60787383-1ecf-48ce-98a6-87f7a2fdacb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:24.953843 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60787383-1ecf-48ce-98a6-87f7a2fdacb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:24.955831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60787383-1ecf-48ce-98a6-87f7a2fdacb7 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:24.982474 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1237/2450] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:58:24 2022] POST /identity/v3/projects => generated 357 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:58:24.986674 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d28a0b-84da-42cf-b1c0-3272a0ef25cc None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:24.997560 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d28a0b-84da-42cf-b1c0-3272a0ef25cc None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:25.057645 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa356c55-466c-495c-b8f0-e1550f3c0af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:25.058272 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa356c55-466c-495c-b8f0-e1550f3c0af6 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:25.058552 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa356c55-466c-495c-b8f0-e1550f3c0af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:25.058828 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa356c55-466c-495c-b8f0-e1550f3c0af6 admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:25.059507 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa356c55-466c-495c-b8f0-e1550f3c0af6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:25.061487 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa356c55-466c-495c-b8f0-e1550f3c0af6 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:25.083217 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1214/2451] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:58:24 2022] POST /identity/v3/users => generated 285 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:58:25.087343 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60787383-1ecf-48ce-98a6-87f7a2fdacb7 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:25.097767 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60787383-1ecf-48ce-98a6-87f7a2fdacb7 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:25.102422 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7d77f-2b11-4f44-a4c8-5a9e0bda94bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:25.103026 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7d77f-2b11-4f44-a4c8-5a9e0bda94bd admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:25.103309 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7d77f-2b11-4f44-a4c8-5a9e0bda94bd admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:25.103582 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7d77f-2b11-4f44-a4c8-5a9e0bda94bd admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:25.104315 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c7d77f-2b11-4f44-a4c8-5a9e0bda94bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:25.105923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c7d77f-2b11-4f44-a4c8-5a9e0bda94bd admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:25.117181 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1238/2452] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:58:25.121255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa356c55-466c-495c-b8f0-e1550f3c0af6 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:25.131466 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa356c55-466c-495c-b8f0-e1550f3c0af6 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:25.136626 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c69e0fe-401c-43a3-8699-1e353b337284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:25.137274 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c69e0fe-401c-43a3-8699-1e353b337284 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:25.137552 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c69e0fe-401c-43a3-8699-1e353b337284 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:25.137824 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c69e0fe-401c-43a3-8699-1e353b337284 admin admin] PATH_INFO: `/v3/projects/22ae7f2ef58f4e539f13a2f570d3fca6/users/465eb951e50f4dfdaef4716f475c33a8/roles/0e78444e43c744668178971af4087ca0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:25.153003 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c69e0fe-401c-43a3-8699-1e353b337284 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22ae7f2ef58f4e539f13a2f570d3fca6, user_id=465eb951e50f4dfdaef4716f475c33a8, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:25.155671 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c69e0fe-401c-43a3-8699-1e353b337284 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:25.173007 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1215/2453] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:58:25 2022] PUT /identity/v3/projects/22ae7f2ef58f4e539f13a2f570d3fca6/users/465eb951e50f4dfdaef4716f475c33a8/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:58:25.176953 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7d77f-2b11-4f44-a4c8-5a9e0bda94bd admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:25.191233 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7d77f-2b11-4f44-a4c8-5a9e0bda94bd admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:25.251224 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270d4d59-58d9-4654-b3e3-9639c881f923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:25.252009 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d4d59-58d9-4654-b3e3-9639c881f923 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:25.252309 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d4d59-58d9-4654-b3e3-9639c881f923 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:25.252732 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d4d59-58d9-4654-b3e3-9639c881f923 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:25.253469 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270d4d59-58d9-4654-b3e3-9639c881f923 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:25.255287 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270d4d59-58d9-4654-b3e3-9639c881f923 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:25.266839 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1239/2454] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:58:25.271034 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c69e0fe-401c-43a3-8699-1e353b337284 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:25.281796 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c69e0fe-401c-43a3-8699-1e353b337284 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:25.287436 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcd4e39-8f0a-43ed-a113-d5c130f5a49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:25.288773 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd4e39-8f0a-43ed-a113-d5c130f5a49a admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:25.289150 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd4e39-8f0a-43ed-a113-d5c130f5a49a admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:25.289487 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd4e39-8f0a-43ed-a113-d5c130f5a49a admin admin] PATH_INFO: `/v3/projects/22ae7f2ef58f4e539f13a2f570d3fca6/users/465eb951e50f4dfdaef4716f475c33a8/roles/0e78444e43c744668178971af4087ca0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:25.291648 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcd4e39-8f0a-43ed-a113-d5c130f5a49a admin admin] RBAC: Authorizing `identity:create_grant(project_id=22ae7f2ef58f4e539f13a2f570d3fca6, user_id=465eb951e50f4dfdaef4716f475c33a8, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:25.294367 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcd4e39-8f0a-43ed-a113-d5c130f5a49a admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:25.306313 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1216/2455] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:58:25 2022] PUT /identity/v3/projects/22ae7f2ef58f4e539f13a2f570d3fca6/users/465eb951e50f4dfdaef4716f475c33a8/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:58:25.317708 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac854270-801b-4f62-9f74-9faf45f08555 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:25.317994 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac854270-801b-4f62-9f74-9faf45f08555 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:25.318264 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac854270-801b-4f62-9f74-9faf45f08555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:25.345328 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-ac854270-801b-4f62-9f74-9faf45f08555 None None] MFA Rules not processed for user `465eb951e50f4dfdaef4716f475c33a8`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:58:25.346619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-ac854270-801b-4f62-9f74-9faf45f08555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:25.416483 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1240/2456] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:58:25 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:58:25.427752 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcd4e39-8f0a-43ed-a113-d5c130f5a49a admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:25.430188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-8dcd4e39-8f0a-43ed-a113-d5c130f5a49a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:25.452296 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:58:25.490609 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcd4e39-8f0a-43ed-a113-d5c130f5a49a admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:25.495137 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e797c-0332-45f3-8b8c-7bf26b777281 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:25.495793 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e797c-0332-45f3-8b8c-7bf26b777281 service cinder] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:25.496077 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e797c-0332-45f3-8b8c-7bf26b777281 service cinder] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:25.496353 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e797c-0332-45f3-8b8c-7bf26b777281 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:25.507427 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e797c-0332-45f3-8b8c-7bf26b777281 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:25.508955 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e797c-0332-45f3-8b8c-7bf26b777281 service cinder] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:25.520531 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1217/2457] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:58:25 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 12:58:25.613559 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac854270-801b-4f62-9f74-9faf45f08555 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:25.624720 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac854270-801b-4f62-9f74-9faf45f08555 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:25.628701 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04a1fbf-42f9-4423-8342-b831125d0237 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:25.630151 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04a1fbf-42f9-4423-8342-b831125d0237 service cinder] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:25.630151 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04a1fbf-42f9-4423-8342-b831125d0237 service cinder] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:25.630151 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04a1fbf-42f9-4423-8342-b831125d0237 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:25.637254 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:58:25.708904 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04a1fbf-42f9-4423-8342-b831125d0237 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:25.710393 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04a1fbf-42f9-4423-8342-b831125d0237 service cinder] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:25.719711 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1241/2458] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:58:25 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:58:35.514748 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=17 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:58:35.715492 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=24 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:58:40.636789 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e797c-0332-45f3-8b8c-7bf26b777281 service cinder] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:40.697525 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e797c-0332-45f3-8b8c-7bf26b777281 service cinder] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:40.702205 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ce218-03a0-41b5-833e-eaed4375ea14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:40.702833 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ce218-03a0-41b5-833e-eaed4375ea14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:40.703108 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ce218-03a0-41b5-833e-eaed4375ea14 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:40.703377 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ce218-03a0-41b5-833e-eaed4375ea14 admin admin] PATH_INFO: `/v3/users/c96ea8a9941540cfa66bde62c66e377c` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:40.704604 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489ce218-03a0-41b5-833e-eaed4375ea14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c96ea8a9941540cfa66bde62c66e377c)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:40.706601 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489ce218-03a0-41b5-833e-eaed4375ea14 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:40.751807 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-489ce218-03a0-41b5-833e-eaed4375ea14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c96ea8a9941540cfa66bde62c66e377c'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:40.752448 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-489ce218-03a0-41b5-833e-eaed4375ea14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c96ea8a9941540cfa66bde62c66e377c'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:40.753041 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-489ce218-03a0-41b5-833e-eaed4375ea14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c96ea8a9941540cfa66bde62c66e377c'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:40.765630 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-489ce218-03a0-41b5-833e-eaed4375ea14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c96ea8a9941540cfa66bde62c66e377c'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:40.772346 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-489ce218-03a0-41b5-833e-eaed4375ea14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c96ea8a9941540cfa66bde62c66e377c'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:40.784897 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1218/2459] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:58:40 2022] DELETE /identity/v3/users/c96ea8a9941540cfa66bde62c66e377c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:58:40.789615 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fdab09-e252-43e2-b760-184c22ae8382 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:40.790140 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fdab09-e252-43e2-b760-184c22ae8382 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:40.790140 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fdab09-e252-43e2-b760-184c22ae8382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:40.809301 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-61fdab09-e252-43e2-b760-184c22ae8382 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:58:40.810485 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-61fdab09-e252-43e2-b760-184c22ae8382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:40.868211 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1242/2460] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:58:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:58:40.878715 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ce218-03a0-41b5-833e-eaed4375ea14 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:40.880161 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-489ce218-03a0-41b5-833e-eaed4375ea14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:40.938134 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ce218-03a0-41b5-833e-eaed4375ea14 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:40.941865 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74c7045-f8c3-4418-903b-58959438dd7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:40.942477 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74c7045-f8c3-4418-903b-58959438dd7d service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:40.942756 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74c7045-f8c3-4418-903b-58959438dd7d service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:40.943033 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74c7045-f8c3-4418-903b-58959438dd7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:40.952670 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74c7045-f8c3-4418-903b-58959438dd7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:40.954036 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74c7045-f8c3-4418-903b-58959438dd7d service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:40.963468 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1219/2461] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:58:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:58:41.237456 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fdab09-e252-43e2-b760-184c22ae8382 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:41.238876 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-61fdab09-e252-43e2-b760-184c22ae8382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:41.250710 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fdab09-e252-43e2-b760-184c22ae8382 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:41.255325 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea6c391-42f8-483a-b658-636dadcd81f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:41.256243 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea6c391-42f8-483a-b658-636dadcd81f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:41.256522 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea6c391-42f8-483a-b658-636dadcd81f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:41.256795 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea6c391-42f8-483a-b658-636dadcd81f1 admin admin] PATH_INFO: `/v3/projects/9bc992a06b094a1daa87cde412e8315e` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:41.258436 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea6c391-42f8-483a-b658-636dadcd81f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bc992a06b094a1daa87cde412e8315e)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:41.260397 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea6c391-42f8-483a-b658-636dadcd81f1 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:41.307432 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-4ea6c391-42f8-483a-b658-636dadcd81f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bc992a06b094a1daa87cde412e8315e', 'request_id': 'req-4ea6c391-42f8-483a-b658-636dadcd81f1'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:41.313729 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-4ea6c391-42f8-483a-b658-636dadcd81f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bc992a06b094a1daa87cde412e8315e', 'request_id': 'req-4ea6c391-42f8-483a-b658-636dadcd81f1'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:41.321802 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-4ea6c391-42f8-483a-b658-636dadcd81f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bc992a06b094a1daa87cde412e8315e', 'request_id': 'req-4ea6c391-42f8-483a-b658-636dadcd81f1'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:41.333497 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-4ea6c391-42f8-483a-b658-636dadcd81f1 admin admin] The token cache is being invalidate because project 9bc992a06b094a1daa87cde412e8315e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:58:41.333790 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-4ea6c391-42f8-483a-b658-636dadcd81f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:41.334421 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-4ea6c391-42f8-483a-b658-636dadcd81f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:41.336027 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1243/2462] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:58:41 2022] DELETE /identity/v3/projects/9bc992a06b094a1daa87cde412e8315e => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:58:41.340666 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74c7045-f8c3-4418-903b-58959438dd7d service neutron] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:41.350297 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74c7045-f8c3-4418-903b-58959438dd7d service neutron] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:41.368799 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:58:41.409619 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e5b47b-5acb-4fd0-b9c7-5409f0280a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:41.410256 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5b47b-5acb-4fd0-b9c7-5409f0280a16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:41.410538 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5b47b-5acb-4fd0-b9c7-5409f0280a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:41.410934 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5b47b-5acb-4fd0-b9c7-5409f0280a16 admin admin] PATH_INFO: `/v3/users/465eb951e50f4dfdaef4716f475c33a8` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:41.412154 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e5b47b-5acb-4fd0-b9c7-5409f0280a16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=465eb951e50f4dfdaef4716f475c33a8)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:41.414101 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e5b47b-5acb-4fd0-b9c7-5409f0280a16 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:41.461204 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-32e5b47b-5acb-4fd0-b9c7-5409f0280a16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '465eb951e50f4dfdaef4716f475c33a8'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:41.461868 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-32e5b47b-5acb-4fd0-b9c7-5409f0280a16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '465eb951e50f4dfdaef4716f475c33a8'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:41.462484 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-32e5b47b-5acb-4fd0-b9c7-5409f0280a16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '465eb951e50f4dfdaef4716f475c33a8'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:41.475258 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-32e5b47b-5acb-4fd0-b9c7-5409f0280a16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '465eb951e50f4dfdaef4716f475c33a8'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:41.490492 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-32e5b47b-5acb-4fd0-b9c7-5409f0280a16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '465eb951e50f4dfdaef4716f475c33a8'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:41.506153 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1220/2463] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:58:41 2022] DELETE /identity/v3/users/465eb951e50f4dfdaef4716f475c33a8 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:58:41.685579 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea6c391-42f8-483a-b658-636dadcd81f1 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:41.687003 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-4ea6c391-42f8-483a-b658-636dadcd81f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:41.748379 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea6c391-42f8-483a-b658-636dadcd81f1 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:41.791106 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:58:41.806340 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6b66ea-aa0c-456b-a15a-72fbf892ce2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:41.807137 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6b66ea-aa0c-456b-a15a-72fbf892ce2d service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:41.807675 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6b66ea-aa0c-456b-a15a-72fbf892ce2d service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:41.807962 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6b66ea-aa0c-456b-a15a-72fbf892ce2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:41.810393 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-5d6b66ea-aa0c-456b-a15a-72fbf892ce2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:41.821560 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e5b47b-5acb-4fd0-b9c7-5409f0280a16 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:41.822885 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-32e5b47b-5acb-4fd0-b9c7-5409f0280a16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:41.883899 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6b66ea-aa0c-456b-a15a-72fbf892ce2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:41.885351 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6b66ea-aa0c-456b-a15a-72fbf892ce2d service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:41.894582 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e5b47b-5acb-4fd0-b9c7-5409f0280a16 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:41.895849 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1244/2464] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:58:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:58:41.900060 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7090768a-a5e3-4f4f-bf08-ba3b7d3c517c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:41.900716 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090768a-a5e3-4f4f-bf08-ba3b7d3c517c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:41.900998 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090768a-a5e3-4f4f-bf08-ba3b7d3c517c admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:41.901274 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090768a-a5e3-4f4f-bf08-ba3b7d3c517c admin admin] PATH_INFO: `/v3/projects/22ae7f2ef58f4e539f13a2f570d3fca6` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:41.902651 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7090768a-a5e3-4f4f-bf08-ba3b7d3c517c admin admin] RBAC: Authorizing `identity:delete_project(project_id=22ae7f2ef58f4e539f13a2f570d3fca6)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:41.906028 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7090768a-a5e3-4f4f-bf08-ba3b7d3c517c admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:41.950972 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:58:41.956482 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-7090768a-a5e3-4f4f-bf08-ba3b7d3c517c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22ae7f2ef58f4e539f13a2f570d3fca6', 'request_id': 'req-7090768a-a5e3-4f4f-bf08-ba3b7d3c517c'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:41.964936 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-7090768a-a5e3-4f4f-bf08-ba3b7d3c517c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22ae7f2ef58f4e539f13a2f570d3fca6', 'request_id': 'req-7090768a-a5e3-4f4f-bf08-ba3b7d3c517c'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:41.975533 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-7090768a-a5e3-4f4f-bf08-ba3b7d3c517c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22ae7f2ef58f4e539f13a2f570d3fca6', 'request_id': 'req-7090768a-a5e3-4f4f-bf08-ba3b7d3c517c'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:41.989715 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-7090768a-a5e3-4f4f-bf08-ba3b7d3c517c admin admin] The token cache is being invalidate because project 22ae7f2ef58f4e539f13a2f570d3fca6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:58:41.989922 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-7090768a-a5e3-4f4f-bf08-ba3b7d3c517c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:41.990598 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-7090768a-a5e3-4f4f-bf08-ba3b7d3c517c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:41.992319 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1221/2465] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:58:41 2022] DELETE /identity/v3/projects/22ae7f2ef58f4e539f13a2f570d3fca6 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:58:41.998014 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae4501-bbcd-4db0-9be7-991a30291aeb None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:41.998297 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae4501-bbcd-4db0-9be7-991a30291aeb None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:41.998592 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae4501-bbcd-4db0-9be7-991a30291aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:42.020003 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-f4ae4501-bbcd-4db0-9be7-991a30291aeb None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:58:42.021205 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-f4ae4501-bbcd-4db0-9be7-991a30291aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:42.089265 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1245/2466] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:58:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:58:42.097294 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e26af-a9a6-4c5a-a1d6-6119cedf2ac9 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:42.097675 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e26af-a9a6-4c5a-a1d6-6119cedf2ac9 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:42.098022 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e26af-a9a6-4c5a-a1d6-6119cedf2ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:42.118608 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-8d2e26af-a9a6-4c5a-a1d6-6119cedf2ac9 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:58:42.119963 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-8d2e26af-a9a6-4c5a-a1d6-6119cedf2ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:42.128441 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1222/2467] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:58:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:58:42.133669 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ae4501-bbcd-4db0-9be7-991a30291aeb None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:42.144057 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ae4501-bbcd-4db0-9be7-991a30291aeb None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:42.149277 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d8ab02-4ad8-4766-94d4-a8c92cea2979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:42.149904 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d8ab02-4ad8-4766-94d4-a8c92cea2979 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:42.150188 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d8ab02-4ad8-4766-94d4-a8c92cea2979 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:42.150461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d8ab02-4ad8-4766-94d4-a8c92cea2979 admin admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:42.151170 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d8ab02-4ad8-4766-94d4-a8c92cea2979 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:58:42.151475 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d8ab02-4ad8-4766-94d4-a8c92cea2979 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:42.153149 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d8ab02-4ad8-4766-94d4-a8c92cea2979 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:42.174228 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1246/2468] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:58:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:58:42.175354 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e26af-a9a6-4c5a-a1d6-6119cedf2ac9 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:42.185762 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e26af-a9a6-4c5a-a1d6-6119cedf2ac9 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:42.190522 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef722b7-2bb3-423f-bcec-784a386b0571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:42.191141 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef722b7-2bb3-423f-bcec-784a386b0571 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:42.191419 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef722b7-2bb3-423f-bcec-784a386b0571 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:42.191756 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef722b7-2bb3-423f-bcec-784a386b0571 admin admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:42.192434 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef722b7-2bb3-423f-bcec-784a386b0571 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:42.194899 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef722b7-2bb3-423f-bcec-784a386b0571 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:42.228594 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1223/2469] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:58:42 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:58:42.230163 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d8ab02-4ad8-4766-94d4-a8c92cea2979 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:42.240796 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d8ab02-4ad8-4766-94d4-a8c92cea2979 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:42.311170 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f7821d-f6fd-4840-a6ec-1d8c96d3f33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:42.311846 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f7821d-f6fd-4840-a6ec-1d8c96d3f33e admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:42.312133 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f7821d-f6fd-4840-a6ec-1d8c96d3f33e admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:42.312409 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f7821d-f6fd-4840-a6ec-1d8c96d3f33e admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:42.313125 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f7821d-f6fd-4840-a6ec-1d8c96d3f33e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:42.315089 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f7821d-f6fd-4840-a6ec-1d8c96d3f33e admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:42.342248 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1247/2470] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:58:42 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:58:42.343740 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef722b7-2bb3-423f-bcec-784a386b0571 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:42.354131 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef722b7-2bb3-423f-bcec-784a386b0571 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:42.359271 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a4757-b2e9-420a-a8ad-101898a78f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:42.359953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a4757-b2e9-420a-a8ad-101898a78f9f admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:42.360259 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a4757-b2e9-420a-a8ad-101898a78f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:42.360514 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a4757-b2e9-420a-a8ad-101898a78f9f admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:42.361219 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3a4757-b2e9-420a-a8ad-101898a78f9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:42.362832 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3a4757-b2e9-420a-a8ad-101898a78f9f admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:42.374344 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1224/2471] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:58:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:58:42.378429 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f7821d-f6fd-4840-a6ec-1d8c96d3f33e admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:42.389742 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f7821d-f6fd-4840-a6ec-1d8c96d3f33e admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:42.394700 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7192896e-5889-4089-bc8d-967c054e9d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:42.395339 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7192896e-5889-4089-bc8d-967c054e9d4b admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:42.395623 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7192896e-5889-4089-bc8d-967c054e9d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:42.395927 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7192896e-5889-4089-bc8d-967c054e9d4b admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:42.396704 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7192896e-5889-4089-bc8d-967c054e9d4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:42.398341 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7192896e-5889-4089-bc8d-967c054e9d4b admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:42.400146 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:58:42.412001 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1248/2472] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:58:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 12:58:42.416288 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a4757-b2e9-420a-a8ad-101898a78f9f admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:42.426322 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a4757-b2e9-420a-a8ad-101898a78f9f admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:42.431557 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915700b1-f5f4-4a45-a2b1-2ec5c330ad5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:42.432332 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915700b1-f5f4-4a45-a2b1-2ec5c330ad5b admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:42.432681 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915700b1-f5f4-4a45-a2b1-2ec5c330ad5b admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:42.433024 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915700b1-f5f4-4a45-a2b1-2ec5c330ad5b admin admin] PATH_INFO: `/v3/projects/537a1139fed842e4959620fbc4ea9560/users/702a53d631434b58aa44794d88310695/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:42.448456 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915700b1-f5f4-4a45-a2b1-2ec5c330ad5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=537a1139fed842e4959620fbc4ea9560, user_id=702a53d631434b58aa44794d88310695, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:42.451174 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915700b1-f5f4-4a45-a2b1-2ec5c330ad5b admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:42.465018 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1225/2473] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:58:42 2022] PUT /identity/v3/projects/537a1139fed842e4959620fbc4ea9560/users/702a53d631434b58aa44794d88310695/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:58:42.477792 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5cdfdf-07e9-42e1-b571-022f51b523cd None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:42.477792 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5cdfdf-07e9-42e1-b571-022f51b523cd None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:42.478263 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5cdfdf-07e9-42e1-b571-022f51b523cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:42.506779 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-4c5cdfdf-07e9-42e1-b571-022f51b523cd None None] MFA Rules not processed for user `702a53d631434b58aa44794d88310695`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:58:42.508044 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-4c5cdfdf-07e9-42e1-b571-022f51b523cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:42.571874 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1249/2474] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:58:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:58:42.576367 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915700b1-f5f4-4a45-a2b1-2ec5c330ad5b admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:42.642466 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915700b1-f5f4-4a45-a2b1-2ec5c330ad5b admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:42.647136 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247d6e8-003a-4317-bee5-9f49c8a218f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:42.647896 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247d6e8-003a-4317-bee5-9f49c8a218f1 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:42.648182 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247d6e8-003a-4317-bee5-9f49c8a218f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:42.648495 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247d6e8-003a-4317-bee5-9f49c8a218f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:42.649201 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c247d6e8-003a-4317-bee5-9f49c8a218f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:42.651103 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c247d6e8-003a-4317-bee5-9f49c8a218f1 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:42.680838 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1226/2475] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:58:42 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:58:42.684929 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5cdfdf-07e9-42e1-b571-022f51b523cd None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:42.696073 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5cdfdf-07e9-42e1-b571-022f51b523cd None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:42.755548 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53361594-f8d1-4294-9ec3-4e412aa7dd65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:42.756531 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53361594-f8d1-4294-9ec3-4e412aa7dd65 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:42.756753 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53361594-f8d1-4294-9ec3-4e412aa7dd65 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:42.757034 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53361594-f8d1-4294-9ec3-4e412aa7dd65 admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:42.757701 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53361594-f8d1-4294-9ec3-4e412aa7dd65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:42.759838 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53361594-f8d1-4294-9ec3-4e412aa7dd65 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:42.784863 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1250/2476] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:58:42 2022] POST /identity/v3/users => generated 300 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:58:42.786613 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247d6e8-003a-4317-bee5-9f49c8a218f1 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:42.797347 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247d6e8-003a-4317-bee5-9f49c8a218f1 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:42.801961 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc45a2-c5fd-41fb-84f5-bf93e8330570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:42.802568 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc45a2-c5fd-41fb-84f5-bf93e8330570 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:42.802848 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc45a2-c5fd-41fb-84f5-bf93e8330570 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:42.803120 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc45a2-c5fd-41fb-84f5-bf93e8330570 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:42.803841 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc45a2-c5fd-41fb-84f5-bf93e8330570 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:42.806099 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc45a2-c5fd-41fb-84f5-bf93e8330570 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:42.818838 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1227/2477] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:58:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:58:42.823044 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53361594-f8d1-4294-9ec3-4e412aa7dd65 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:42.833258 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53361594-f8d1-4294-9ec3-4e412aa7dd65 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:42.838809 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ea0496-58bd-49bc-9e43-2016dc8ee7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:42.839417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea0496-58bd-49bc-9e43-2016dc8ee7bb admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:42.839730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea0496-58bd-49bc-9e43-2016dc8ee7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:42.840008 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea0496-58bd-49bc-9e43-2016dc8ee7bb admin admin] PATH_INFO: `/v3/projects/99ce02933e444c578feb1eb28ac359be/users/6d6d26ee4bfe4d508c3c6ead6391fae7/roles/0e78444e43c744668178971af4087ca0` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:42.855418 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ea0496-58bd-49bc-9e43-2016dc8ee7bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=99ce02933e444c578feb1eb28ac359be, user_id=6d6d26ee4bfe4d508c3c6ead6391fae7, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:42.858158 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ea0496-58bd-49bc-9e43-2016dc8ee7bb admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:42.872425 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1251/2478] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:58:42 2022] PUT /identity/v3/projects/99ce02933e444c578feb1eb28ac359be/users/6d6d26ee4bfe4d508c3c6ead6391fae7/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:58:42.876679 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc45a2-c5fd-41fb-84f5-bf93e8330570 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:42.889182 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc45a2-c5fd-41fb-84f5-bf93e8330570 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:42.934661 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:58:42.952682 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d448f-78bb-4f29-a58e-8d1dd4840889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:42.953427 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d448f-78bb-4f29-a58e-8d1dd4840889 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:42.953712 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d448f-78bb-4f29-a58e-8d1dd4840889 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:42.953989 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d448f-78bb-4f29-a58e-8d1dd4840889 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:42.954686 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19d448f-78bb-4f29-a58e-8d1dd4840889 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:42.956337 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19d448f-78bb-4f29-a58e-8d1dd4840889 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:42.969848 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1228/2479] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:58:42 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:58:42.971777 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ea0496-58bd-49bc-9e43-2016dc8ee7bb admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:42.983542 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ea0496-58bd-49bc-9e43-2016dc8ee7bb admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:42.988346 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b047d-d8c1-4d5f-975c-617ad72367c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:42.989019 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b047d-d8c1-4d5f-975c-617ad72367c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:42.989305 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b047d-d8c1-4d5f-975c-617ad72367c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:42.989584 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b047d-d8c1-4d5f-975c-617ad72367c3 admin admin] PATH_INFO: `/v3/projects/99ce02933e444c578feb1eb28ac359be/users/6d6d26ee4bfe4d508c3c6ead6391fae7/roles/0e78444e43c744668178971af4087ca0` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:42.991813 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896b047d-d8c1-4d5f-975c-617ad72367c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99ce02933e444c578feb1eb28ac359be, user_id=6d6d26ee4bfe4d508c3c6ead6391fae7, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:42.994495 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896b047d-d8c1-4d5f-975c-617ad72367c3 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:43.014383 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1252/2480] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:58:42 2022] PUT /identity/v3/projects/99ce02933e444c578feb1eb28ac359be/users/6d6d26ee4bfe4d508c3c6ead6391fae7/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:58:43.015992 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33fee13-6f1a-49b6-b263-b3001f537163 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:43.016360 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33fee13-6f1a-49b6-b263-b3001f537163 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:43.016705 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33fee13-6f1a-49b6-b263-b3001f537163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:43.048072 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-d33fee13-6f1a-49b6-b263-b3001f537163 None None] MFA Rules not processed for user `6d6d26ee4bfe4d508c3c6ead6391fae7`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:58:43.049327 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-d33fee13-6f1a-49b6-b263-b3001f537163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:43.135192 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1229/2481] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:58:43 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:58:43.146785 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b047d-d8c1-4d5f-975c-617ad72367c3 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:43.148577 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-896b047d-d8c1-4d5f-975c-617ad72367c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:43.209178 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:58:43.230287 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b047d-d8c1-4d5f-975c-617ad72367c3 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:43.234313 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a231e933-d92b-443d-9791-479799bccc46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:43.235024 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231e933-d92b-443d-9791-479799bccc46 service cinder] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:43.235382 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231e933-d92b-443d-9791-479799bccc46 service cinder] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:43.235763 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231e933-d92b-443d-9791-479799bccc46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:43.246355 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a231e933-d92b-443d-9791-479799bccc46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:43.247943 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a231e933-d92b-443d-9791-479799bccc46 service cinder] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:43.259956 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1253/2482] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:58:43 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:58:43.335040 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33fee13-6f1a-49b6-b263-b3001f537163 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:43.406750 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33fee13-6f1a-49b6-b263-b3001f537163 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:43.411428 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50011ba-15bf-42ab-a6ce-b5ec969c8fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:43.412092 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50011ba-15bf-42ab-a6ce-b5ec969c8fe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:43.412393 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50011ba-15bf-42ab-a6ce-b5ec969c8fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:43.412653 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50011ba-15bf-42ab-a6ce-b5ec969c8fe6 admin admin] PATH_INFO: `/v3/users/702a53d631434b58aa44794d88310695` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:43.413829 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50011ba-15bf-42ab-a6ce-b5ec969c8fe6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=702a53d631434b58aa44794d88310695)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:43.416500 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50011ba-15bf-42ab-a6ce-b5ec969c8fe6 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:43.465132 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-b50011ba-15bf-42ab-a6ce-b5ec969c8fe6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '702a53d631434b58aa44794d88310695'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:43.465903 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-b50011ba-15bf-42ab-a6ce-b5ec969c8fe6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '702a53d631434b58aa44794d88310695'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:43.466512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-b50011ba-15bf-42ab-a6ce-b5ec969c8fe6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '702a53d631434b58aa44794d88310695'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:43.479616 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-b50011ba-15bf-42ab-a6ce-b5ec969c8fe6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '702a53d631434b58aa44794d88310695'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:43.480850 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:58:43.487518 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-b50011ba-15bf-42ab-a6ce-b5ec969c8fe6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '702a53d631434b58aa44794d88310695'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:43.501898 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1230/2483] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:58:43 2022] DELETE /identity/v3/users/702a53d631434b58aa44794d88310695 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:58:43.507721 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd42b2-357b-4338-aa8e-347df38ce421 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:43.508239 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd42b2-357b-4338-aa8e-347df38ce421 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:43.508519 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd42b2-357b-4338-aa8e-347df38ce421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:43.527834 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-e2dd42b2-357b-4338-aa8e-347df38ce421 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:58:43.528971 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-e2dd42b2-357b-4338-aa8e-347df38ce421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:43.599396 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1254/2484] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:58:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:58:43.609904 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50011ba-15bf-42ab-a6ce-b5ec969c8fe6 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:43.611682 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-b50011ba-15bf-42ab-a6ce-b5ec969c8fe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:43.682783 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50011ba-15bf-42ab-a6ce-b5ec969c8fe6 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:43.687215 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1812f907-8e48-4e91-aa18-2997c2119627 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:43.688217 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812f907-8e48-4e91-aa18-2997c2119627 service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:43.688217 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812f907-8e48-4e91-aa18-2997c2119627 service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:43.689099 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812f907-8e48-4e91-aa18-2997c2119627 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:43.701694 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1812f907-8e48-4e91-aa18-2997c2119627 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:43.703372 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1812f907-8e48-4e91-aa18-2997c2119627 service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:43.714861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1231/2485] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:58:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 12:58:44.025838 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dd42b2-357b-4338-aa8e-347df38ce421 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:44.027978 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-e2dd42b2-357b-4338-aa8e-347df38ce421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:44.051308 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dd42b2-357b-4338-aa8e-347df38ce421 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:44.057622 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee531f25-21a7-40b0-8b4a-c36aafe947f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:44.058481 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee531f25-21a7-40b0-8b4a-c36aafe947f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:44.058826 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee531f25-21a7-40b0-8b4a-c36aafe947f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:44.059401 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee531f25-21a7-40b0-8b4a-c36aafe947f9 admin admin] PATH_INFO: `/v3/projects/537a1139fed842e4959620fbc4ea9560` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:44.061372 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee531f25-21a7-40b0-8b4a-c36aafe947f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=537a1139fed842e4959620fbc4ea9560)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:44.064982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee531f25-21a7-40b0-8b4a-c36aafe947f9 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:44.131534 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-ee531f25-21a7-40b0-8b4a-c36aafe947f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '537a1139fed842e4959620fbc4ea9560', 'request_id': 'req-ee531f25-21a7-40b0-8b4a-c36aafe947f9'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:44.139029 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-ee531f25-21a7-40b0-8b4a-c36aafe947f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '537a1139fed842e4959620fbc4ea9560', 'request_id': 'req-ee531f25-21a7-40b0-8b4a-c36aafe947f9'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:44.146535 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-ee531f25-21a7-40b0-8b4a-c36aafe947f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '537a1139fed842e4959620fbc4ea9560', 'request_id': 'req-ee531f25-21a7-40b0-8b4a-c36aafe947f9'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:44.157930 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-ee531f25-21a7-40b0-8b4a-c36aafe947f9 admin admin] The token cache is being invalidate because project 537a1139fed842e4959620fbc4ea9560 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:58:44.158230 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-ee531f25-21a7-40b0-8b4a-c36aafe947f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:44.158896 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-ee531f25-21a7-40b0-8b4a-c36aafe947f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:44.160558 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1255/2486] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:58:44 2022] DELETE /identity/v3/projects/537a1139fed842e4959620fbc4ea9560 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:58:44.166922 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1812f907-8e48-4e91-aa18-2997c2119627 service neutron] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:44.185822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1812f907-8e48-4e91-aa18-2997c2119627 service neutron] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:44.245723 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7535eb-0a38-420e-80d6-561989129d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:44.246363 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7535eb-0a38-420e-80d6-561989129d1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:44.246641 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7535eb-0a38-420e-80d6-561989129d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:44.246919 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7535eb-0a38-420e-80d6-561989129d1e admin admin] PATH_INFO: `/v3/users/6d6d26ee4bfe4d508c3c6ead6391fae7` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:44.248143 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7535eb-0a38-420e-80d6-561989129d1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d6d26ee4bfe4d508c3c6ead6391fae7)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:44.250087 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7535eb-0a38-420e-80d6-561989129d1e admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:44.300150 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-ce7535eb-0a38-420e-80d6-561989129d1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d6d26ee4bfe4d508c3c6ead6391fae7'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:44.300734 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-ce7535eb-0a38-420e-80d6-561989129d1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d6d26ee4bfe4d508c3c6ead6391fae7'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:44.301353 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-ce7535eb-0a38-420e-80d6-561989129d1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d6d26ee4bfe4d508c3c6ead6391fae7'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:44.314238 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-ce7535eb-0a38-420e-80d6-561989129d1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d6d26ee4bfe4d508c3c6ead6391fae7'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:44.321553 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-ce7535eb-0a38-420e-80d6-561989129d1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d6d26ee4bfe4d508c3c6ead6391fae7'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:44.335498 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1232/2487] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:58:44 2022] DELETE /identity/v3/users/6d6d26ee4bfe4d508c3c6ead6391fae7 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:58:44.638683 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee531f25-21a7-40b0-8b4a-c36aafe947f9 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:44.641112 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-ee531f25-21a7-40b0-8b4a-c36aafe947f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:44.666655 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee531f25-21a7-40b0-8b4a-c36aafe947f9 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:44.743137 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f83ae-c329-466e-9674-f9168067f562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:44.744010 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f83ae-c329-466e-9674-f9168067f562 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:44.744132 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f83ae-c329-466e-9674-f9168067f562 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:44.744363 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f83ae-c329-466e-9674-f9168067f562 admin admin] PATH_INFO: `/v3/projects/99ce02933e444c578feb1eb28ac359be` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:44.745552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f83ae-c329-466e-9674-f9168067f562 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99ce02933e444c578feb1eb28ac359be)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:44.747488 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f83ae-c329-466e-9674-f9168067f562 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:44.756394 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:58:44.811463 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-ea0f83ae-c329-466e-9674-f9168067f562 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99ce02933e444c578feb1eb28ac359be', 'request_id': 'req-ea0f83ae-c329-466e-9674-f9168067f562'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:44.817840 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-ea0f83ae-c329-466e-9674-f9168067f562 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99ce02933e444c578feb1eb28ac359be', 'request_id': 'req-ea0f83ae-c329-466e-9674-f9168067f562'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:44.826255 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-ea0f83ae-c329-466e-9674-f9168067f562 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99ce02933e444c578feb1eb28ac359be', 'request_id': 'req-ea0f83ae-c329-466e-9674-f9168067f562'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:44.842109 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-ea0f83ae-c329-466e-9674-f9168067f562 admin admin] The token cache is being invalidate because project 99ce02933e444c578feb1eb28ac359be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:58:44.842410 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-ea0f83ae-c329-466e-9674-f9168067f562 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:44.843131 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-ea0f83ae-c329-466e-9674-f9168067f562 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:44.844994 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1256/2488] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:58:44 2022] DELETE /identity/v3/projects/99ce02933e444c578feb1eb28ac359be => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:58:44.849648 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae218b1a-cd42-4fd0-a258-48b998544893 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:44.850003 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae218b1a-cd42-4fd0-a258-48b998544893 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:44.850794 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae218b1a-cd42-4fd0-a258-48b998544893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:44.861330 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:58:44.890133 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-ae218b1a-cd42-4fd0-a258-48b998544893 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:58:44.891044 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-ae218b1a-cd42-4fd0-a258-48b998544893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:44.959667 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1233/2489] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:58:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:58:44.968255 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba04976e-fbf8-4e79-9d14-2c7b71750a87 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:44.968552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba04976e-fbf8-4e79-9d14-2c7b71750a87 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:44.969202 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba04976e-fbf8-4e79-9d14-2c7b71750a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:44.999282 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-ba04976e-fbf8-4e79-9d14-2c7b71750a87 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:58:45.000651 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-ba04976e-fbf8-4e79-9d14-2c7b71750a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:45.015680 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae218b1a-cd42-4fd0-a258-48b998544893 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:45.018930 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1257/2490] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:58:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:58:45.027565 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae218b1a-cd42-4fd0-a258-48b998544893 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:45.035288 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf01c3-9685-4736-990a-06d37561fff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:45.036265 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf01c3-9685-4736-990a-06d37561fff4 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:45.036596 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf01c3-9685-4736-990a-06d37561fff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:45.037495 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf01c3-9685-4736-990a-06d37561fff4 admin admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:45.037648 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabf01c3-9685-4736-990a-06d37561fff4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:58:45.038110 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabf01c3-9685-4736-990a-06d37561fff4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:45.040035 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabf01c3-9685-4736-990a-06d37561fff4 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:45.054171 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1234/2491] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:58:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:58:45.058956 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba04976e-fbf8-4e79-9d14-2c7b71750a87 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:45.070093 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba04976e-fbf8-4e79-9d14-2c7b71750a87 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:45.077111 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af1cc1e-793d-4ea9-b160-5967657cb4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:45.078320 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1cc1e-793d-4ea9-b160-5967657cb4a8 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:45.078611 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1cc1e-793d-4ea9-b160-5967657cb4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:45.078883 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1cc1e-793d-4ea9-b160-5967657cb4a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:45.079751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af1cc1e-793d-4ea9-b160-5967657cb4a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:45.083291 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af1cc1e-793d-4ea9-b160-5967657cb4a8 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:45.127624 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1258/2492] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:58:45 2022] POST /identity/v3/projects => generated 371 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:58:45.129096 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf01c3-9685-4736-990a-06d37561fff4 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:45.154357 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf01c3-9685-4736-990a-06d37561fff4 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:45.239069 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e10dd5-e0a0-41a7-a9ec-b7c00a2b5c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:45.239744 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e10dd5-e0a0-41a7-a9ec-b7c00a2b5c88 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:45.240031 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e10dd5-e0a0-41a7-a9ec-b7c00a2b5c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:45.240308 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e10dd5-e0a0-41a7-a9ec-b7c00a2b5c88 admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:45.241263 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e10dd5-e0a0-41a7-a9ec-b7c00a2b5c88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:45.243817 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e10dd5-e0a0-41a7-a9ec-b7c00a2b5c88 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:45.272694 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1235/2493] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:58:45 2022] POST /identity/v3/users => generated 286 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:58:45.274723 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af1cc1e-793d-4ea9-b160-5967657cb4a8 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:45.295374 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af1cc1e-793d-4ea9-b160-5967657cb4a8 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:45.301997 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44526a5d-799c-4ede-8173-71d87e5b3a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:45.302641 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44526a5d-799c-4ede-8173-71d87e5b3a2d admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:45.302926 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44526a5d-799c-4ede-8173-71d87e5b3a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:45.303199 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44526a5d-799c-4ede-8173-71d87e5b3a2d admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:45.303938 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44526a5d-799c-4ede-8173-71d87e5b3a2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:45.306039 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44526a5d-799c-4ede-8173-71d87e5b3a2d admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:45.311067 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e10dd5-e0a0-41a7-a9ec-b7c00a2b5c88 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:45.312994 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-56e10dd5-e0a0-41a7-a9ec-b7c00a2b5c88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:45.318057 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1259/2494] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:58:45.323223 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44526a5d-799c-4ede-8173-71d87e5b3a2d admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:45.327274 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e10dd5-e0a0-41a7-a9ec-b7c00a2b5c88 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:45.332315 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5491e84-5f77-4e84-b6e8-dfd3cbb9621b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:45.332964 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5491e84-5f77-4e84-b6e8-dfd3cbb9621b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:45.333274 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5491e84-5f77-4e84-b6e8-dfd3cbb9621b admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:45.333548 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5491e84-5f77-4e84-b6e8-dfd3cbb9621b admin admin] PATH_INFO: `/v3/users/17b7e2c1b1b04366b99e9ab892aeb835` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:45.334701 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44526a5d-799c-4ede-8173-71d87e5b3a2d admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:45.334822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5491e84-5f77-4e84-b6e8-dfd3cbb9621b admin admin] RBAC: Authorizing `identity:delete_user(user_id=17b7e2c1b1b04366b99e9ab892aeb835)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:45.336931 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5491e84-5f77-4e84-b6e8-dfd3cbb9621b admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:45.339351 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eefa5fd-6168-4eb3-8f13-fdecde72027b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:45.340017 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eefa5fd-6168-4eb3-8f13-fdecde72027b admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:45.340303 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eefa5fd-6168-4eb3-8f13-fdecde72027b admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:45.340761 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eefa5fd-6168-4eb3-8f13-fdecde72027b admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:45.341341 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eefa5fd-6168-4eb3-8f13-fdecde72027b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:45.343197 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eefa5fd-6168-4eb3-8f13-fdecde72027b admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:45.359261 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1260/2495] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 12:58:45.360377 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eefa5fd-6168-4eb3-8f13-fdecde72027b admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:45.373802 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eefa5fd-6168-4eb3-8f13-fdecde72027b admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:45.379368 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bf867c-8e28-4ed7-9a2d-92123de31fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:45.380088 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf867c-8e28-4ed7-9a2d-92123de31fa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:45.380373 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf867c-8e28-4ed7-9a2d-92123de31fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:45.380651 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf867c-8e28-4ed7-9a2d-92123de31fa4 admin admin] PATH_INFO: `/v3/projects/dc0f5db491bd494d943de6bf1084c1a5/users/8e8b5c62a961409baaaa101f61437f1e/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:45.393446 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-c5491e84-5f77-4e84-b6e8-dfd3cbb9621b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17b7e2c1b1b04366b99e9ab892aeb835'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:45.394369 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-c5491e84-5f77-4e84-b6e8-dfd3cbb9621b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17b7e2c1b1b04366b99e9ab892aeb835'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:45.395373 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-c5491e84-5f77-4e84-b6e8-dfd3cbb9621b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17b7e2c1b1b04366b99e9ab892aeb835'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:45.402963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bf867c-8e28-4ed7-9a2d-92123de31fa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc0f5db491bd494d943de6bf1084c1a5, user_id=8e8b5c62a961409baaaa101f61437f1e, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:45.406101 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bf867c-8e28-4ed7-9a2d-92123de31fa4 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:45.410632 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-c5491e84-5f77-4e84-b6e8-dfd3cbb9621b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17b7e2c1b1b04366b99e9ab892aeb835'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:45.418245 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-c5491e84-5f77-4e84-b6e8-dfd3cbb9621b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17b7e2c1b1b04366b99e9ab892aeb835'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:45.421011 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:58:45.424546 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1261/2496] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:58:45 2022] PUT /identity/v3/projects/dc0f5db491bd494d943de6bf1084c1a5/users/8e8b5c62a961409baaaa101f61437f1e/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:58:45.430490 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882820c-657f-446c-b0f2-ad98732e9697 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:45.430961 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882820c-657f-446c-b0f2-ad98732e9697 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:45.431234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882820c-657f-446c-b0f2-ad98732e9697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:45.439821 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1236/2497] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:58:45 2022] DELETE /identity/v3/users/17b7e2c1b1b04366b99e9ab892aeb835 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:58:45.456568 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-0882820c-657f-446c-b0f2-ad98732e9697 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:58:45.457807 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-0882820c-657f-446c-b0f2-ad98732e9697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:45.532689 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1262/2498] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:58:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:58:45.542993 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5491e84-5f77-4e84-b6e8-dfd3cbb9621b admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:45.544443 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-c5491e84-5f77-4e84-b6e8-dfd3cbb9621b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:45.603038 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5491e84-5f77-4e84-b6e8-dfd3cbb9621b admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:45.606652 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b80b9a-a189-44c7-b818-4644c8d55af1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:45.607265 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b80b9a-a189-44c7-b818-4644c8d55af1 service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:45.607543 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b80b9a-a189-44c7-b818-4644c8d55af1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:45.607842 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b80b9a-a189-44c7-b818-4644c8d55af1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:45.618026 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b80b9a-a189-44c7-b818-4644c8d55af1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:45.619780 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b80b9a-a189-44c7-b818-4644c8d55af1 service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:45.631390 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1237/2499] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:58:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:58:45.631547 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882820c-657f-446c-b0f2-ad98732e9697 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:45.633211 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-0882820c-657f-446c-b0f2-ad98732e9697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:45.652944 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882820c-657f-446c-b0f2-ad98732e9697 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:45.658600 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615546c-1fa1-4697-a5a5-4de4c09327ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:45.659264 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615546c-1fa1-4697-a5a5-4de4c09327ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:45.659614 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615546c-1fa1-4697-a5a5-4de4c09327ae admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:45.659915 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615546c-1fa1-4697-a5a5-4de4c09327ae admin admin] PATH_INFO: `/v3/projects/a93dd858fb7a40fbbb679fbe676b1f3c` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:45.661140 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d615546c-1fa1-4697-a5a5-4de4c09327ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=a93dd858fb7a40fbbb679fbe676b1f3c)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:45.663089 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d615546c-1fa1-4697-a5a5-4de4c09327ae admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:45.682533 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:58:45.719703 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-d615546c-1fa1-4697-a5a5-4de4c09327ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a93dd858fb7a40fbbb679fbe676b1f3c', 'request_id': 'req-d615546c-1fa1-4697-a5a5-4de4c09327ae'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:45.727622 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-d615546c-1fa1-4697-a5a5-4de4c09327ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a93dd858fb7a40fbbb679fbe676b1f3c', 'request_id': 'req-d615546c-1fa1-4697-a5a5-4de4c09327ae'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:45.735472 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-d615546c-1fa1-4697-a5a5-4de4c09327ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a93dd858fb7a40fbbb679fbe676b1f3c', 'request_id': 'req-d615546c-1fa1-4697-a5a5-4de4c09327ae'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:45.747210 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-d615546c-1fa1-4697-a5a5-4de4c09327ae admin admin] The token cache is being invalidate because project a93dd858fb7a40fbbb679fbe676b1f3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:58:45.747498 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-d615546c-1fa1-4697-a5a5-4de4c09327ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:45.748194 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-d615546c-1fa1-4697-a5a5-4de4c09327ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:45.749969 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1263/2500] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:58:45 2022] DELETE /identity/v3/projects/a93dd858fb7a40fbbb679fbe676b1f3c => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:58:45.754969 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d372f5-899a-4cb5-83c1-3881e61ea80d None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:45.755260 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d372f5-899a-4cb5-83c1-3881e61ea80d None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:45.755535 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d372f5-899a-4cb5-83c1-3881e61ea80d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:45.776474 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-91d372f5-899a-4cb5-83c1-3881e61ea80d None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:58:45.777692 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-91d372f5-899a-4cb5-83c1-3881e61ea80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:45.843722 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1238/2501] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:58:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:58:45.851435 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec74bd-998f-45af-bc0a-b98ebec0d702 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:45.852237 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec74bd-998f-45af-bc0a-b98ebec0d702 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:45.852516 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec74bd-998f-45af-bc0a-b98ebec0d702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:45.874736 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-ddec74bd-998f-45af-bc0a-b98ebec0d702 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:58:45.876016 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-ddec74bd-998f-45af-bc0a-b98ebec0d702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:45.884879 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1264/2502] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:58:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:58:45.889184 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d372f5-899a-4cb5-83c1-3881e61ea80d None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:45.899955 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d372f5-899a-4cb5-83c1-3881e61ea80d None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:45.905905 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04083446-56b6-4db4-8439-079002d04b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:45.906528 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04083446-56b6-4db4-8439-079002d04b74 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:45.906810 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04083446-56b6-4db4-8439-079002d04b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:45.907084 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04083446-56b6-4db4-8439-079002d04b74 admin admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:45.907819 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04083446-56b6-4db4-8439-079002d04b74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:58:45.908121 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04083446-56b6-4db4-8439-079002d04b74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:45.909727 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04083446-56b6-4db4-8439-079002d04b74 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:45.925840 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1239/2503] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:58:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:58:45.930885 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec74bd-998f-45af-bc0a-b98ebec0d702 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:45.942232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec74bd-998f-45af-bc0a-b98ebec0d702 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:45.949752 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d5b01-8f41-447a-802b-60bed3eb0c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:45.950590 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d5b01-8f41-447a-802b-60bed3eb0c5b admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:45.950967 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d5b01-8f41-447a-802b-60bed3eb0c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:45.951339 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d5b01-8f41-447a-802b-60bed3eb0c5b admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:45.952508 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743d5b01-8f41-447a-802b-60bed3eb0c5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:45.954312 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743d5b01-8f41-447a-802b-60bed3eb0c5b admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:45.995120 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1265/2504] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:58:45 2022] POST /identity/v3/projects => generated 385 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:58:45.996612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04083446-56b6-4db4-8439-079002d04b74 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:46.008844 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04083446-56b6-4db4-8439-079002d04b74 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:46.072729 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c36bb9-1c87-4064-815e-bb56e990add2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:46.073447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c36bb9-1c87-4064-815e-bb56e990add2 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:46.073798 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c36bb9-1c87-4064-815e-bb56e990add2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:46.074197 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c36bb9-1c87-4064-815e-bb56e990add2 admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:46.075858 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c36bb9-1c87-4064-815e-bb56e990add2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:46.077863 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c36bb9-1c87-4064-815e-bb56e990add2 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:46.106188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1240/2505] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:58:45 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:58:46.107617 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d5b01-8f41-447a-802b-60bed3eb0c5b admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:46.118456 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d5b01-8f41-447a-802b-60bed3eb0c5b admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:46.124946 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5359a801-8338-45b1-9a30-f5e163565c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:46.125686 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359a801-8338-45b1-9a30-f5e163565c9a admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:46.125971 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359a801-8338-45b1-9a30-f5e163565c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:46.126275 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359a801-8338-45b1-9a30-f5e163565c9a admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:46.126958 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5359a801-8338-45b1-9a30-f5e163565c9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:46.128658 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5359a801-8338-45b1-9a30-f5e163565c9a admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:46.140169 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1266/2506] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:58:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:58:46.144190 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c36bb9-1c87-4064-815e-bb56e990add2 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:46.155028 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c36bb9-1c87-4064-815e-bb56e990add2 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:46.159756 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f304372c-813d-4ba1-9768-342894b2d457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:46.160824 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f304372c-813d-4ba1-9768-342894b2d457 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:46.160824 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f304372c-813d-4ba1-9768-342894b2d457 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:46.161069 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f304372c-813d-4ba1-9768-342894b2d457 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:46.161603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f304372c-813d-4ba1-9768-342894b2d457 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:46.163567 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f304372c-813d-4ba1-9768-342894b2d457 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:46.180774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1241/2507] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:58:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:58:46.185079 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5359a801-8338-45b1-9a30-f5e163565c9a admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:46.195606 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5359a801-8338-45b1-9a30-f5e163565c9a admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:46.200243 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0ae423-bd25-4a5e-b089-0512c6b915c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:46.200911 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0ae423-bd25-4a5e-b089-0512c6b915c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:46.201192 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0ae423-bd25-4a5e-b089-0512c6b915c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:46.201465 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0ae423-bd25-4a5e-b089-0512c6b915c7 admin admin] PATH_INFO: `/v3/projects/cf380877d2614eae80ecc2d8530bfa99/users/3b3d1dc6079d415b8eaafca6ec1cacc4/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:46.216789 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0ae423-bd25-4a5e-b089-0512c6b915c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf380877d2614eae80ecc2d8530bfa99, user_id=3b3d1dc6079d415b8eaafca6ec1cacc4, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:46.219474 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0ae423-bd25-4a5e-b089-0512c6b915c7 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:46.234733 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1267/2508] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:58:46 2022] PUT /identity/v3/projects/cf380877d2614eae80ecc2d8530bfa99/users/3b3d1dc6079d415b8eaafca6ec1cacc4/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:58:46.244716 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac77f3-39af-43e2-b2ec-f46b3ed28045 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:46.245156 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac77f3-39af-43e2-b2ec-f46b3ed28045 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:46.245428 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac77f3-39af-43e2-b2ec-f46b3ed28045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:46.254053 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:58:46.280376 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-dfac77f3-39af-43e2-b2ec-f46b3ed28045 None None] MFA Rules not processed for user `3b3d1dc6079d415b8eaafca6ec1cacc4`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:58:46.280994 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-dfac77f3-39af-43e2-b2ec-f46b3ed28045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:46.305670 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade3d52-9199-4260-80da-042abd20c2eb None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:46.305958 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade3d52-9199-4260-80da-042abd20c2eb None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:46.306238 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade3d52-9199-4260-80da-042abd20c2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:46.338262 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-aade3d52-9199-4260-80da-042abd20c2eb None None] MFA Rules not processed for user `8e8b5c62a961409baaaa101f61437f1e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:58:46.339386 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-aade3d52-9199-4260-80da-042abd20c2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:46.368891 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1242/2509] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:58:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:58:46.379462 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfac77f3-39af-43e2-b2ec-f46b3ed28045 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:46.382068 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-dfac77f3-39af-43e2-b2ec-f46b3ed28045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:46.423762 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1268/2510] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:58:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:58:46.439310 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade3d52-9199-4260-80da-042abd20c2eb None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:46.440981 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-aade3d52-9199-4260-80da-042abd20c2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:46.471417 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfac77f3-39af-43e2-b2ec-f46b3ed28045 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:46.477220 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:58:46.480129 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76f012e-0761-4412-af1d-6cd4ab87bc1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:46.480755 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f012e-0761-4412-af1d-6cd4ab87bc1b service nova] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:46.481030 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f012e-0761-4412-af1d-6cd4ab87bc1b service nova] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:46.481296 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f012e-0761-4412-af1d-6cd4ab87bc1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:46.491469 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76f012e-0761-4412-af1d-6cd4ab87bc1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:46.494446 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76f012e-0761-4412-af1d-6cd4ab87bc1b service nova] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:46.508303 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade3d52-9199-4260-80da-042abd20c2eb None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:46.510283 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1243/2511] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:58:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 12:58:46.512442 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef19fc3-bec7-4ce9-b70f-cb9305899971 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:46.513171 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef19fc3-bec7-4ce9-b70f-cb9305899971 service cinder] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:46.513521 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef19fc3-bec7-4ce9-b70f-cb9305899971 service cinder] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:46.513862 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef19fc3-bec7-4ce9-b70f-cb9305899971 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:46.523128 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef19fc3-bec7-4ce9-b70f-cb9305899971 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:46.525856 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef19fc3-bec7-4ce9-b70f-cb9305899971 service cinder] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:46.544687 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1269/2512] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:58:46 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:58:47.951709 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3052f-c4b6-4c20-9f64-321e6d850454 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:47.952177 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3052f-c4b6-4c20-9f64-321e6d850454 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:47.952293 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3052f-c4b6-4c20-9f64-321e6d850454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:47.971392 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-bfb3052f-c4b6-4c20-9f64-321e6d850454 None None] MFA Rules not processed for user `8e8b5c62a961409baaaa101f61437f1e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:58:47.972968 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-bfb3052f-c4b6-4c20-9f64-321e6d850454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:47.982276 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1244/2513] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:58:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:58:53.894818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef19fc3-bec7-4ce9-b70f-cb9305899971 service cinder] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:53.957207 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef19fc3-bec7-4ce9-b70f-cb9305899971 service cinder] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:53.961826 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eda3f7-5140-4374-841d-b6925b2f72fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:53.962454 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eda3f7-5140-4374-841d-b6925b2f72fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:53.962736 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eda3f7-5140-4374-841d-b6925b2f72fa admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:53.963012 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eda3f7-5140-4374-841d-b6925b2f72fa admin admin] PATH_INFO: `/v3/users/3b3d1dc6079d415b8eaafca6ec1cacc4` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:53.964197 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eda3f7-5140-4374-841d-b6925b2f72fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b3d1dc6079d415b8eaafca6ec1cacc4)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:53.966141 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eda3f7-5140-4374-841d-b6925b2f72fa admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:54.012768 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-a8eda3f7-5140-4374-841d-b6925b2f72fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b3d1dc6079d415b8eaafca6ec1cacc4'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:54.019981 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-a8eda3f7-5140-4374-841d-b6925b2f72fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b3d1dc6079d415b8eaafca6ec1cacc4'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:54.020595 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-a8eda3f7-5140-4374-841d-b6925b2f72fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b3d1dc6079d415b8eaafca6ec1cacc4'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:54.027562 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-a8eda3f7-5140-4374-841d-b6925b2f72fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b3d1dc6079d415b8eaafca6ec1cacc4'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:54.041370 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-a8eda3f7-5140-4374-841d-b6925b2f72fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b3d1dc6079d415b8eaafca6ec1cacc4'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:54.047534 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1270/2514] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:58:53 2022] DELETE /identity/v3/users/3b3d1dc6079d415b8eaafca6ec1cacc4 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:58:54.052188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cecc55-8f19-43f6-87cf-c7f05d552178 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:54.052679 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cecc55-8f19-43f6-87cf-c7f05d552178 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:54.052784 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cecc55-8f19-43f6-87cf-c7f05d552178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:54.072304 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-04cecc55-8f19-43f6-87cf-c7f05d552178 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:58:54.073509 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-04cecc55-8f19-43f6-87cf-c7f05d552178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:54.137217 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1245/2515] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:58:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:58:54.147391 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eda3f7-5140-4374-841d-b6925b2f72fa admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:54.148995 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-a8eda3f7-5140-4374-841d-b6925b2f72fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:54.220475 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eda3f7-5140-4374-841d-b6925b2f72fa admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:54.224304 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88daa68-5c51-418e-b0ee-4f648690df12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:54.224924 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88daa68-5c51-418e-b0ee-4f648690df12 service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:54.225205 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88daa68-5c51-418e-b0ee-4f648690df12 service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:54.225478 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88daa68-5c51-418e-b0ee-4f648690df12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:54.232221 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:58:54.237027 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88daa68-5c51-418e-b0ee-4f648690df12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:54.238846 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88daa68-5c51-418e-b0ee-4f648690df12 service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:54.249800 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1271/2516] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:58:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:58:54.960445 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cecc55-8f19-43f6-87cf-c7f05d552178 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:54.961965 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-04cecc55-8f19-43f6-87cf-c7f05d552178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:54.973827 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cecc55-8f19-43f6-87cf-c7f05d552178 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:54.978710 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5233bd88-a62c-46fa-a2f7-382e5873e61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:54.979213 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233bd88-a62c-46fa-a2f7-382e5873e61f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:54.979491 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233bd88-a62c-46fa-a2f7-382e5873e61f admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:54.979793 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233bd88-a62c-46fa-a2f7-382e5873e61f admin admin] PATH_INFO: `/v3/projects/cf380877d2614eae80ecc2d8530bfa99` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:54.981207 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5233bd88-a62c-46fa-a2f7-382e5873e61f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf380877d2614eae80ecc2d8530bfa99)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:54.983460 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5233bd88-a62c-46fa-a2f7-382e5873e61f admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:54.991358 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:58:55.035971 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-5233bd88-a62c-46fa-a2f7-382e5873e61f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf380877d2614eae80ecc2d8530bfa99', 'request_id': 'req-5233bd88-a62c-46fa-a2f7-382e5873e61f'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:55.043026 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-5233bd88-a62c-46fa-a2f7-382e5873e61f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf380877d2614eae80ecc2d8530bfa99', 'request_id': 'req-5233bd88-a62c-46fa-a2f7-382e5873e61f'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:55.050581 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-5233bd88-a62c-46fa-a2f7-382e5873e61f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf380877d2614eae80ecc2d8530bfa99', 'request_id': 'req-5233bd88-a62c-46fa-a2f7-382e5873e61f'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:55.061257 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-5233bd88-a62c-46fa-a2f7-382e5873e61f admin admin] The token cache is being invalidate because project cf380877d2614eae80ecc2d8530bfa99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:58:55.061837 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-5233bd88-a62c-46fa-a2f7-382e5873e61f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:55.062157 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-5233bd88-a62c-46fa-a2f7-382e5873e61f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:55.063693 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1246/2517] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:58:54 2022] DELETE /identity/v3/projects/cf380877d2614eae80ecc2d8530bfa99 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:58:55.068666 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224cfacf-e3a7-4e2b-b8a2-681e979c23d4 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:55.069092 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224cfacf-e3a7-4e2b-b8a2-681e979c23d4 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:55.069199 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224cfacf-e3a7-4e2b-b8a2-681e979c23d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:55.094221 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-224cfacf-e3a7-4e2b-b8a2-681e979c23d4 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:58:55.095445 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-224cfacf-e3a7-4e2b-b8a2-681e979c23d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:55.154606 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1272/2518] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:58:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:58:55.163419 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df9777-a556-4221-8dc1-6af104533389 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:55.163740 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df9777-a556-4221-8dc1-6af104533389 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:55.164020 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df9777-a556-4221-8dc1-6af104533389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:55.184114 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-27df9777-a556-4221-8dc1-6af104533389 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:58:55.185310 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-27df9777-a556-4221-8dc1-6af104533389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:55.193146 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1247/2519] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:58:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:58:55.198264 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224cfacf-e3a7-4e2b-b8a2-681e979c23d4 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:55.208576 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224cfacf-e3a7-4e2b-b8a2-681e979c23d4 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:55.213672 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ff32a-ffbc-49de-9fe8-349180014361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:55.214273 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ff32a-ffbc-49de-9fe8-349180014361 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:55.214553 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ff32a-ffbc-49de-9fe8-349180014361 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:55.214826 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ff32a-ffbc-49de-9fe8-349180014361 admin admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:55.215516 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67ff32a-ffbc-49de-9fe8-349180014361 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:58:55.215848 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67ff32a-ffbc-49de-9fe8-349180014361 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:55.217495 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67ff32a-ffbc-49de-9fe8-349180014361 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:55.229021 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1273/2520] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:58:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:58:55.233428 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df9777-a556-4221-8dc1-6af104533389 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:55.242583 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df9777-a556-4221-8dc1-6af104533389 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:55.247135 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa349c5-e806-43df-a4d5-04f6dbf8f80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:55.247769 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa349c5-e806-43df-a4d5-04f6dbf8f80a admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:55.248043 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa349c5-e806-43df-a4d5-04f6dbf8f80a admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:55.248311 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa349c5-e806-43df-a4d5-04f6dbf8f80a admin admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:55.248964 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa349c5-e806-43df-a4d5-04f6dbf8f80a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:55.250877 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa349c5-e806-43df-a4d5-04f6dbf8f80a admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:55.278543 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1248/2521] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:58:55 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:58:55.282738 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ff32a-ffbc-49de-9fe8-349180014361 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:55.292798 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ff32a-ffbc-49de-9fe8-349180014361 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:55.349093 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9292a2ec-f760-4746-9f61-2da279eb4143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:55.349703 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9292a2ec-f760-4746-9f61-2da279eb4143 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:55.349985 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9292a2ec-f760-4746-9f61-2da279eb4143 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:55.350257 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9292a2ec-f760-4746-9f61-2da279eb4143 admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:55.350913 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9292a2ec-f760-4746-9f61-2da279eb4143 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:55.352904 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9292a2ec-f760-4746-9f61-2da279eb4143 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:55.374105 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1274/2522] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:58:55 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:58:55.378217 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa349c5-e806-43df-a4d5-04f6dbf8f80a admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:55.389683 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa349c5-e806-43df-a4d5-04f6dbf8f80a admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:55.394440 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245c5e5-daa9-4b3d-a1fd-2f986fdbd51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:55.395043 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245c5e5-daa9-4b3d-a1fd-2f986fdbd51c admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:55.395326 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245c5e5-daa9-4b3d-a1fd-2f986fdbd51c admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:55.395602 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245c5e5-daa9-4b3d-a1fd-2f986fdbd51c admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:55.396336 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d245c5e5-daa9-4b3d-a1fd-2f986fdbd51c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:55.397946 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d245c5e5-daa9-4b3d-a1fd-2f986fdbd51c admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:55.408602 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1249/2523] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:58:55.412736 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9292a2ec-f760-4746-9f61-2da279eb4143 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:55.422659 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9292a2ec-f760-4746-9f61-2da279eb4143 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:55.427475 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84f7026-c196-41b4-b910-f1feb5d0c62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:55.428119 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84f7026-c196-41b4-b910-f1feb5d0c62e admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:55.428406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84f7026-c196-41b4-b910-f1feb5d0c62e admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:55.428704 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84f7026-c196-41b4-b910-f1feb5d0c62e admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:55.429425 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84f7026-c196-41b4-b910-f1feb5d0c62e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:55.431385 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84f7026-c196-41b4-b910-f1feb5d0c62e admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:55.443492 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1275/2524] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 12:58:55.448335 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245c5e5-daa9-4b3d-a1fd-2f986fdbd51c admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:55.458440 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245c5e5-daa9-4b3d-a1fd-2f986fdbd51c admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:55.463781 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa3679-5389-442d-a555-3bbbb52b1d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:55.464370 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa3679-5389-442d-a555-3bbbb52b1d72 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:55.464647 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa3679-5389-442d-a555-3bbbb52b1d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:55.464919 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa3679-5389-442d-a555-3bbbb52b1d72 admin admin] PATH_INFO: `/v3/projects/d9ac013dc2d845298225c52cbf24e3ac/users/ae62d026b2a14ba1ba37628f2d4933db/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:55.480428 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa3679-5389-442d-a555-3bbbb52b1d72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9ac013dc2d845298225c52cbf24e3ac, user_id=ae62d026b2a14ba1ba37628f2d4933db, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:55.483312 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa3679-5389-442d-a555-3bbbb52b1d72 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:55.498085 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1250/2525] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:58:55 2022] PUT /identity/v3/projects/d9ac013dc2d845298225c52cbf24e3ac/users/ae62d026b2a14ba1ba37628f2d4933db/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:58:55.509178 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8c930-86dc-449e-b4c0-83a8e592a416 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:55.509709 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8c930-86dc-449e-b4c0-83a8e592a416 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:55.510157 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8c930-86dc-449e-b4c0-83a8e592a416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:55.539807 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-50f8c930-86dc-449e-b4c0-83a8e592a416 None None] MFA Rules not processed for user `ae62d026b2a14ba1ba37628f2d4933db`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:58:55.540899 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-50f8c930-86dc-449e-b4c0-83a8e592a416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:55.591574 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:58:55.611266 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1276/2526] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:58:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:58:55.615769 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa3679-5389-442d-a555-3bbbb52b1d72 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:55.678657 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa3679-5389-442d-a555-3bbbb52b1d72 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:55.684253 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abd2b42-fd3a-49b1-ac43-08bee6f8d717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:55.684864 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd2b42-fd3a-49b1-ac43-08bee6f8d717 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:55.685150 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd2b42-fd3a-49b1-ac43-08bee6f8d717 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:55.685428 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd2b42-fd3a-49b1-ac43-08bee6f8d717 admin admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:55.686092 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abd2b42-fd3a-49b1-ac43-08bee6f8d717 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:55.688052 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abd2b42-fd3a-49b1-ac43-08bee6f8d717 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:55.718846 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1251/2527] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:58:55 2022] POST /identity/v3/projects => generated 381 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:58:55.720038 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f8c930-86dc-449e-b4c0-83a8e592a416 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:55.731080 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f8c930-86dc-449e-b4c0-83a8e592a416 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:55.794328 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc4ec7-fd7c-483c-b0a9-fe6bc1e5fdc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:55.795057 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc4ec7-fd7c-483c-b0a9-fe6bc1e5fdc8 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:55.795340 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc4ec7-fd7c-483c-b0a9-fe6bc1e5fdc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:55.795615 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc4ec7-fd7c-483c-b0a9-fe6bc1e5fdc8 admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:55.796343 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fc4ec7-fd7c-483c-b0a9-fe6bc1e5fdc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:55.798413 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fc4ec7-fd7c-483c-b0a9-fe6bc1e5fdc8 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:55.822278 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1277/2528] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:58:55 2022] POST /identity/v3/users => generated 297 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:58:55.826420 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abd2b42-fd3a-49b1-ac43-08bee6f8d717 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:55.837205 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abd2b42-fd3a-49b1-ac43-08bee6f8d717 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:55.842112 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec0e60-3eb5-4d30-a2fd-ddd1f184ba9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:55.842828 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec0e60-3eb5-4d30-a2fd-ddd1f184ba9a admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:55.843031 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec0e60-3eb5-4d30-a2fd-ddd1f184ba9a admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:55.843358 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec0e60-3eb5-4d30-a2fd-ddd1f184ba9a admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:55.844088 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec0e60-3eb5-4d30-a2fd-ddd1f184ba9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:55.845734 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec0e60-3eb5-4d30-a2fd-ddd1f184ba9a admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:55.857509 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1252/2529] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:58:55.861730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc4ec7-fd7c-483c-b0a9-fe6bc1e5fdc8 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:55.873078 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc4ec7-fd7c-483c-b0a9-fe6bc1e5fdc8 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:55.878299 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cded3f-632a-4e42-91b0-44e0061c4713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:55.878981 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cded3f-632a-4e42-91b0-44e0061c4713 admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:55.879267 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cded3f-632a-4e42-91b0-44e0061c4713 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:55.879544 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cded3f-632a-4e42-91b0-44e0061c4713 admin admin] PATH_INFO: `/v3/projects/2feff6da544940a8aad4513195e84a00/users/45ad86dd2efc4d779d16e6838f0c8c35/roles/0e78444e43c744668178971af4087ca0` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:55.896859 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cded3f-632a-4e42-91b0-44e0061c4713 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2feff6da544940a8aad4513195e84a00, user_id=45ad86dd2efc4d779d16e6838f0c8c35, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:55.899879 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cded3f-632a-4e42-91b0-44e0061c4713 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:55.915249 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1278/2530] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:58:55 2022] PUT /identity/v3/projects/2feff6da544940a8aad4513195e84a00/users/45ad86dd2efc4d779d16e6838f0c8c35/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:58:55.919405 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec0e60-3eb5-4d30-a2fd-ddd1f184ba9a admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:55.925856 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:58:55.931144 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec0e60-3eb5-4d30-a2fd-ddd1f184ba9a admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:56.000836 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac60b2-f66a-4c06-9b70-d202a67df199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:56.001578 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac60b2-f66a-4c06-9b70-d202a67df199 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:56.001772 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac60b2-f66a-4c06-9b70-d202a67df199 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:56.002040 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac60b2-f66a-4c06-9b70-d202a67df199 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:56.002849 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ac60b2-f66a-4c06-9b70-d202a67df199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:56.004738 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ac60b2-f66a-4c06-9b70-d202a67df199 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:56.015557 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1253/2531] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 12:58:56.019778 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cded3f-632a-4e42-91b0-44e0061c4713 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:56.030394 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cded3f-632a-4e42-91b0-44e0061c4713 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:56.035019 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98515f15-7a8e-4825-a211-0ef5dcb43189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:56.035877 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98515f15-7a8e-4825-a211-0ef5dcb43189 admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:56.036105 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98515f15-7a8e-4825-a211-0ef5dcb43189 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:56.036300 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98515f15-7a8e-4825-a211-0ef5dcb43189 admin admin] PATH_INFO: `/v3/projects/2feff6da544940a8aad4513195e84a00/users/45ad86dd2efc4d779d16e6838f0c8c35/roles/0e78444e43c744668178971af4087ca0` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:56.038962 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98515f15-7a8e-4825-a211-0ef5dcb43189 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2feff6da544940a8aad4513195e84a00, user_id=45ad86dd2efc4d779d16e6838f0c8c35, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:56.043922 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98515f15-7a8e-4825-a211-0ef5dcb43189 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:56.058733 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1279/2532] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:58:56 2022] PUT /identity/v3/projects/2feff6da544940a8aad4513195e84a00/users/45ad86dd2efc4d779d16e6838f0c8c35/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:58:56.069947 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb87002-26cd-4a44-a044-6eb6d95e31e0 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:56.070468 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb87002-26cd-4a44-a044-6eb6d95e31e0 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:56.070631 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb87002-26cd-4a44-a044-6eb6d95e31e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:56.100516 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-3fb87002-26cd-4a44-a044-6eb6d95e31e0 None None] MFA Rules not processed for user `45ad86dd2efc4d779d16e6838f0c8c35`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:58:56.101751 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-3fb87002-26cd-4a44-a044-6eb6d95e31e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:56.172201 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1254/2533] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:58:56 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:58:56.187112 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98515f15-7a8e-4825-a211-0ef5dcb43189 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:56.189037 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-98515f15-7a8e-4825-a211-0ef5dcb43189 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:56.253017 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98515f15-7a8e-4825-a211-0ef5dcb43189 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:56.257259 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d952e-f528-4dd2-8981-c8a328595fdb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:56.257881 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d952e-f528-4dd2-8981-c8a328595fdb service cinder] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:56.258186 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d952e-f528-4dd2-8981-c8a328595fdb service cinder] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:56.258485 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d952e-f528-4dd2-8981-c8a328595fdb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:56.267687 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457d952e-f528-4dd2-8981-c8a328595fdb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:56.269123 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457d952e-f528-4dd2-8981-c8a328595fdb service cinder] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:56.278639 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1280/2534] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:58:56 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:58:56.598186 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb87002-26cd-4a44-a044-6eb6d95e31e0 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:56.607938 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb87002-26cd-4a44-a044-6eb6d95e31e0 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:56.612453 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b510492-deae-43fb-9e2f-b9835b98ebc7 tempest-VolumeQuotaClassesTest-1812580233 tempest-VolumeQuotaClassesTest-1812580233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ad86dd2efc4d779d16e6838f0c8c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2feff6da544940a8aad4513195e84a00', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:56.613055 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b510492-deae-43fb-9e2f-b9835b98ebc7 tempest-VolumeQuotaClassesTest-1812580233 tempest-VolumeQuotaClassesTest-1812580233-project-admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:56.613335 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b510492-deae-43fb-9e2f-b9835b98ebc7 tempest-VolumeQuotaClassesTest-1812580233 tempest-VolumeQuotaClassesTest-1812580233-project-admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:56.613605 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b510492-deae-43fb-9e2f-b9835b98ebc7 tempest-VolumeQuotaClassesTest-1812580233 tempest-VolumeQuotaClassesTest-1812580233-project-admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:56.614290 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b510492-deae-43fb-9e2f-b9835b98ebc7 tempest-VolumeQuotaClassesTest-1812580233 tempest-VolumeQuotaClassesTest-1812580233-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:58:56.614652 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b510492-deae-43fb-9e2f-b9835b98ebc7 tempest-VolumeQuotaClassesTest-1812580233 tempest-VolumeQuotaClassesTest-1812580233-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:56.616275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b510492-deae-43fb-9e2f-b9835b98ebc7 tempest-VolumeQuotaClassesTest-1812580233 tempest-VolumeQuotaClassesTest-1812580233-project-admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:56.627545 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1255/2535] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:58:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:58:56.632176 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d952e-f528-4dd2-8981-c8a328595fdb service cinder] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:56.641904 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d952e-f528-4dd2-8981-c8a328595fdb service cinder] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:56.646523 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e617dc3b-6156-4443-9d1a-33cd80502bbd tempest-VolumeQuotaClassesTest-1812580233 tempest-VolumeQuotaClassesTest-1812580233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ad86dd2efc4d779d16e6838f0c8c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2feff6da544940a8aad4513195e84a00', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:56.647132 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617dc3b-6156-4443-9d1a-33cd80502bbd tempest-VolumeQuotaClassesTest-1812580233 tempest-VolumeQuotaClassesTest-1812580233-project-admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:56.647418 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617dc3b-6156-4443-9d1a-33cd80502bbd tempest-VolumeQuotaClassesTest-1812580233 tempest-VolumeQuotaClassesTest-1812580233-project-admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:56.647718 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617dc3b-6156-4443-9d1a-33cd80502bbd tempest-VolumeQuotaClassesTest-1812580233 tempest-VolumeQuotaClassesTest-1812580233-project-admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:56.648415 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e617dc3b-6156-4443-9d1a-33cd80502bbd tempest-VolumeQuotaClassesTest-1812580233 tempest-VolumeQuotaClassesTest-1812580233-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:56.650846 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e617dc3b-6156-4443-9d1a-33cd80502bbd tempest-VolumeQuotaClassesTest-1812580233 tempest-VolumeQuotaClassesTest-1812580233-project-admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:56.679209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1281/2536] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:58:56 2022] POST /identity/v3/projects => generated 354 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:58:56.683131 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b510492-deae-43fb-9e2f-b9835b98ebc7 tempest-VolumeQuotaClassesTest-1812580233 tempest-VolumeQuotaClassesTest-1812580233-project-admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:56.693373 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b510492-deae-43fb-9e2f-b9835b98ebc7 tempest-VolumeQuotaClassesTest-1812580233 tempest-VolumeQuotaClassesTest-1812580233-project-admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:56.752743 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e649e64f-d4c7-42d6-bb5c-ef38240186d5 tempest-VolumeQuotaClassesTest-1812580233 tempest-VolumeQuotaClassesTest-1812580233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ad86dd2efc4d779d16e6838f0c8c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2feff6da544940a8aad4513195e84a00', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:56.753394 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649e64f-d4c7-42d6-bb5c-ef38240186d5 tempest-VolumeQuotaClassesTest-1812580233 tempest-VolumeQuotaClassesTest-1812580233-project-admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:56.753822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649e64f-d4c7-42d6-bb5c-ef38240186d5 tempest-VolumeQuotaClassesTest-1812580233 tempest-VolumeQuotaClassesTest-1812580233-project-admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:56.754108 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649e64f-d4c7-42d6-bb5c-ef38240186d5 tempest-VolumeQuotaClassesTest-1812580233 tempest-VolumeQuotaClassesTest-1812580233-project-admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:56.754811 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e649e64f-d4c7-42d6-bb5c-ef38240186d5 tempest-VolumeQuotaClassesTest-1812580233 tempest-VolumeQuotaClassesTest-1812580233-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:58:56.755122 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e649e64f-d4c7-42d6-bb5c-ef38240186d5 tempest-VolumeQuotaClassesTest-1812580233 tempest-VolumeQuotaClassesTest-1812580233-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:56.757020 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e649e64f-d4c7-42d6-bb5c-ef38240186d5 tempest-VolumeQuotaClassesTest-1812580233 tempest-VolumeQuotaClassesTest-1812580233-project-admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:56.769848 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1256/2537] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:58:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:58:56.806170 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e617dc3b-6156-4443-9d1a-33cd80502bbd tempest-VolumeQuotaClassesTest-1812580233 tempest-VolumeQuotaClassesTest-1812580233-project-admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:56.816788 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e617dc3b-6156-4443-9d1a-33cd80502bbd tempest-VolumeQuotaClassesTest-1812580233 tempest-VolumeQuotaClassesTest-1812580233-project-admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:56.821846 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb0749-b3f3-42e4-ba1e-19f73cd0e81c tempest-VolumeQuotaClassesTest-1812580233 tempest-VolumeQuotaClassesTest-1812580233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ad86dd2efc4d779d16e6838f0c8c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2feff6da544940a8aad4513195e84a00', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:56.822475 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb0749-b3f3-42e4-ba1e-19f73cd0e81c tempest-VolumeQuotaClassesTest-1812580233 tempest-VolumeQuotaClassesTest-1812580233-project-admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:56.822757 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb0749-b3f3-42e4-ba1e-19f73cd0e81c tempest-VolumeQuotaClassesTest-1812580233 tempest-VolumeQuotaClassesTest-1812580233-project-admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:56.823029 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb0749-b3f3-42e4-ba1e-19f73cd0e81c tempest-VolumeQuotaClassesTest-1812580233 tempest-VolumeQuotaClassesTest-1812580233-project-admin] PATH_INFO: `/v3/projects/f36fb356a1444301a6762666b81ccc82` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:56.824450 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fb0749-b3f3-42e4-ba1e-19f73cd0e81c tempest-VolumeQuotaClassesTest-1812580233 tempest-VolumeQuotaClassesTest-1812580233-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f36fb356a1444301a6762666b81ccc82)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:56.828063 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fb0749-b3f3-42e4-ba1e-19f73cd0e81c tempest-VolumeQuotaClassesTest-1812580233 tempest-VolumeQuotaClassesTest-1812580233-project-admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:56.851153 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:58:56.881174 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-48fb0749-b3f3-42e4-ba1e-19f73cd0e81c tempest-VolumeQuotaClassesTest-1812580233 tempest-VolumeQuotaClassesTest-1812580233-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f36fb356a1444301a6762666b81ccc82', 'request_id': 'req-48fb0749-b3f3-42e4-ba1e-19f73cd0e81c'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:56.889010 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-48fb0749-b3f3-42e4-ba1e-19f73cd0e81c tempest-VolumeQuotaClassesTest-1812580233 tempest-VolumeQuotaClassesTest-1812580233-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f36fb356a1444301a6762666b81ccc82', 'request_id': 'req-48fb0749-b3f3-42e4-ba1e-19f73cd0e81c'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:56.896895 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-48fb0749-b3f3-42e4-ba1e-19f73cd0e81c tempest-VolumeQuotaClassesTest-1812580233 tempest-VolumeQuotaClassesTest-1812580233-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f36fb356a1444301a6762666b81ccc82', 'request_id': 'req-48fb0749-b3f3-42e4-ba1e-19f73cd0e81c'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:56.912332 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-48fb0749-b3f3-42e4-ba1e-19f73cd0e81c tempest-VolumeQuotaClassesTest-1812580233 tempest-VolumeQuotaClassesTest-1812580233-project-admin] The token cache is being invalidate because project f36fb356a1444301a6762666b81ccc82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:58:56.912626 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-48fb0749-b3f3-42e4-ba1e-19f73cd0e81c tempest-VolumeQuotaClassesTest-1812580233 tempest-VolumeQuotaClassesTest-1812580233-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:56.913308 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-48fb0749-b3f3-42e4-ba1e-19f73cd0e81c tempest-VolumeQuotaClassesTest-1812580233 tempest-VolumeQuotaClassesTest-1812580233-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:56.914839 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1282/2538] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:58:56 2022] DELETE /identity/v3/projects/f36fb356a1444301a6762666b81ccc82 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:58:57.103337 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e649e64f-d4c7-42d6-bb5c-ef38240186d5 tempest-VolumeQuotaClassesTest-1812580233 tempest-VolumeQuotaClassesTest-1812580233-project-admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:57.105105 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-e649e64f-d4c7-42d6-bb5c-ef38240186d5 tempest-VolumeQuotaClassesTest-1812580233 tempest-VolumeQuotaClassesTest-1812580233-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:57.114834 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:58:57.169694 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e649e64f-d4c7-42d6-bb5c-ef38240186d5 tempest-VolumeQuotaClassesTest-1812580233 tempest-VolumeQuotaClassesTest-1812580233-project-admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:57.225419 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528ed8c3-9e0a-4f0c-a572-56a909f445bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:57.226052 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ed8c3-9e0a-4f0c-a572-56a909f445bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:57.226333 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ed8c3-9e0a-4f0c-a572-56a909f445bc admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:57.226609 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ed8c3-9e0a-4f0c-a572-56a909f445bc admin admin] PATH_INFO: `/v3/users/ae62d026b2a14ba1ba37628f2d4933db` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:57.227840 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528ed8c3-9e0a-4f0c-a572-56a909f445bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae62d026b2a14ba1ba37628f2d4933db)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:57.229772 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528ed8c3-9e0a-4f0c-a572-56a909f445bc admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:57.276371 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-528ed8c3-9e0a-4f0c-a572-56a909f445bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae62d026b2a14ba1ba37628f2d4933db'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:57.277023 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-528ed8c3-9e0a-4f0c-a572-56a909f445bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae62d026b2a14ba1ba37628f2d4933db'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:57.277625 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-528ed8c3-9e0a-4f0c-a572-56a909f445bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae62d026b2a14ba1ba37628f2d4933db'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:57.289904 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-528ed8c3-9e0a-4f0c-a572-56a909f445bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae62d026b2a14ba1ba37628f2d4933db'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:57.296690 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-528ed8c3-9e0a-4f0c-a572-56a909f445bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae62d026b2a14ba1ba37628f2d4933db'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:57.310450 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1257/2539] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:58:57 2022] DELETE /identity/v3/users/ae62d026b2a14ba1ba37628f2d4933db => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:58:57.315028 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ca49a-1f4b-44cb-9199-72367aa413ab None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:57.315497 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ca49a-1f4b-44cb-9199-72367aa413ab None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:57.315616 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ca49a-1f4b-44cb-9199-72367aa413ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:57.334684 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-7d7ca49a-1f4b-44cb-9199-72367aa413ab None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:58:57.335850 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-7d7ca49a-1f4b-44cb-9199-72367aa413ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:57.395513 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1283/2540] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:58:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:58:57.406135 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528ed8c3-9e0a-4f0c-a572-56a909f445bc admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:57.407551 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-528ed8c3-9e0a-4f0c-a572-56a909f445bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:57.446763 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:58:57.475914 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528ed8c3-9e0a-4f0c-a572-56a909f445bc admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:57.480097 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15023f26-0cb3-4d07-aef3-28b0ce534643 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:57.480703 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15023f26-0cb3-4d07-aef3-28b0ce534643 service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:57.480981 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15023f26-0cb3-4d07-aef3-28b0ce534643 service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:57.481252 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15023f26-0cb3-4d07-aef3-28b0ce534643 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:57.490130 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15023f26-0cb3-4d07-aef3-28b0ce534643 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:57.491542 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15023f26-0cb3-4d07-aef3-28b0ce534643 service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:57.501589 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1258/2541] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:58:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 12:58:57.772130 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ca49a-1f4b-44cb-9199-72367aa413ab None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:57.774082 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-7d7ca49a-1f4b-44cb-9199-72367aa413ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:57.793567 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ca49a-1f4b-44cb-9199-72367aa413ab None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:57.799672 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee01e41-9ea4-4f43-81dd-968ea6192999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:57.800356 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee01e41-9ea4-4f43-81dd-968ea6192999 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:57.800714 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee01e41-9ea4-4f43-81dd-968ea6192999 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:57.800946 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee01e41-9ea4-4f43-81dd-968ea6192999 admin admin] PATH_INFO: `/v3/projects/d9ac013dc2d845298225c52cbf24e3ac` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:57.802155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee01e41-9ea4-4f43-81dd-968ea6192999 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9ac013dc2d845298225c52cbf24e3ac)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:57.804328 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee01e41-9ea4-4f43-81dd-968ea6192999 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:57.872838 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-fee01e41-9ea4-4f43-81dd-968ea6192999 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9ac013dc2d845298225c52cbf24e3ac', 'request_id': 'req-fee01e41-9ea4-4f43-81dd-968ea6192999'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:57.883559 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-fee01e41-9ea4-4f43-81dd-968ea6192999 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9ac013dc2d845298225c52cbf24e3ac', 'request_id': 'req-fee01e41-9ea4-4f43-81dd-968ea6192999'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:57.892547 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-fee01e41-9ea4-4f43-81dd-968ea6192999 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9ac013dc2d845298225c52cbf24e3ac', 'request_id': 'req-fee01e41-9ea4-4f43-81dd-968ea6192999'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:57.908729 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-fee01e41-9ea4-4f43-81dd-968ea6192999 admin admin] The token cache is being invalidate because project d9ac013dc2d845298225c52cbf24e3ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:58:57.909537 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-fee01e41-9ea4-4f43-81dd-968ea6192999 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:57.910237 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-fee01e41-9ea4-4f43-81dd-968ea6192999 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:57.912596 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1284/2542] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:58:57 2022] DELETE /identity/v3/projects/d9ac013dc2d845298225c52cbf24e3ac => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:58:57.917338 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15023f26-0cb3-4d07-aef3-28b0ce534643 service neutron] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:57.929204 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15023f26-0cb3-4d07-aef3-28b0ce534643 service neutron] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:58.020678 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd0de49-cd8b-4872-9d6e-9332bd79d6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:58.021333 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0de49-cd8b-4872-9d6e-9332bd79d6df admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:58.021614 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0de49-cd8b-4872-9d6e-9332bd79d6df admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:58.021891 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0de49-cd8b-4872-9d6e-9332bd79d6df admin admin] PATH_INFO: `/v3/users/45ad86dd2efc4d779d16e6838f0c8c35` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:58.023215 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd0de49-cd8b-4872-9d6e-9332bd79d6df admin admin] RBAC: Authorizing `identity:delete_user(user_id=45ad86dd2efc4d779d16e6838f0c8c35)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:58.025771 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd0de49-cd8b-4872-9d6e-9332bd79d6df admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:58.074725 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-edd0de49-cd8b-4872-9d6e-9332bd79d6df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45ad86dd2efc4d779d16e6838f0c8c35'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:58.075392 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-edd0de49-cd8b-4872-9d6e-9332bd79d6df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45ad86dd2efc4d779d16e6838f0c8c35'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:58.076045 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-edd0de49-cd8b-4872-9d6e-9332bd79d6df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45ad86dd2efc4d779d16e6838f0c8c35'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:58.089010 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-edd0de49-cd8b-4872-9d6e-9332bd79d6df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45ad86dd2efc4d779d16e6838f0c8c35'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:58.095869 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-edd0de49-cd8b-4872-9d6e-9332bd79d6df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45ad86dd2efc4d779d16e6838f0c8c35'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:58.109932 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1259/2543] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:58:57 2022] DELETE /identity/v3/users/45ad86dd2efc4d779d16e6838f0c8c35 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:58:58.380154 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee01e41-9ea4-4f43-81dd-968ea6192999 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:58.382373 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-fee01e41-9ea4-4f43-81dd-968ea6192999 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:58.397800 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee01e41-9ea4-4f43-81dd-968ea6192999 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:58.431763 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:58:58.464587 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f954ea-1ec9-4725-a74c-e8211da9163a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:58.465224 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f954ea-1ec9-4725-a74c-e8211da9163a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:58.465517 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f954ea-1ec9-4725-a74c-e8211da9163a admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:58.465778 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f954ea-1ec9-4725-a74c-e8211da9163a admin admin] PATH_INFO: `/v3/projects/2feff6da544940a8aad4513195e84a00` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:58.466934 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f954ea-1ec9-4725-a74c-e8211da9163a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2feff6da544940a8aad4513195e84a00)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:58.468914 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f954ea-1ec9-4725-a74c-e8211da9163a admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:58.516056 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-a0f954ea-1ec9-4725-a74c-e8211da9163a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2feff6da544940a8aad4513195e84a00', 'request_id': 'req-a0f954ea-1ec9-4725-a74c-e8211da9163a'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:58.522138 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-a0f954ea-1ec9-4725-a74c-e8211da9163a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2feff6da544940a8aad4513195e84a00', 'request_id': 'req-a0f954ea-1ec9-4725-a74c-e8211da9163a'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:58.530899 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-a0f954ea-1ec9-4725-a74c-e8211da9163a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2feff6da544940a8aad4513195e84a00', 'request_id': 'req-a0f954ea-1ec9-4725-a74c-e8211da9163a'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:58.541267 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-a0f954ea-1ec9-4725-a74c-e8211da9163a admin admin] The token cache is being invalidate because project 2feff6da544940a8aad4513195e84a00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:58:58.541557 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-a0f954ea-1ec9-4725-a74c-e8211da9163a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:58.542179 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-a0f954ea-1ec9-4725-a74c-e8211da9163a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:58:58.543776 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1285/2544] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:58:58 2022] DELETE /identity/v3/projects/2feff6da544940a8aad4513195e84a00 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:58:58.548707 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133ca5dc-b141-4ee1-9973-f3d088d4abbf None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:58.548998 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133ca5dc-b141-4ee1-9973-f3d088d4abbf None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:58.549277 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133ca5dc-b141-4ee1-9973-f3d088d4abbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:58.568634 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-133ca5dc-b141-4ee1-9973-f3d088d4abbf None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:58:58.569830 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-133ca5dc-b141-4ee1-9973-f3d088d4abbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:58.617389 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:58:58.630371 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1260/2545] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:58:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:58:58.640373 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2a474-bb72-4355-ae7a-c805c2f34ae4 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:58.640373 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2a474-bb72-4355-ae7a-c805c2f34ae4 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:58.640373 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2a474-bb72-4355-ae7a-c805c2f34ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:58.658965 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-8de2a474-bb72-4355-ae7a-c805c2f34ae4 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:58:58.660231 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-8de2a474-bb72-4355-ae7a-c805c2f34ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:58.668028 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1286/2546] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:58:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:58:58.672259 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133ca5dc-b141-4ee1-9973-f3d088d4abbf None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:58.682647 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133ca5dc-b141-4ee1-9973-f3d088d4abbf None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:58.687169 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d56e0-5880-48e1-b2db-9d382747394f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:58.687799 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d56e0-5880-48e1-b2db-9d382747394f admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:58.688080 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d56e0-5880-48e1-b2db-9d382747394f admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:58.688353 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d56e0-5880-48e1-b2db-9d382747394f admin admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:58.689073 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d56e0-5880-48e1-b2db-9d382747394f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:58:58.689379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d56e0-5880-48e1-b2db-9d382747394f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:58.691005 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d56e0-5880-48e1-b2db-9d382747394f admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:58.703157 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1261/2547] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:58:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:58:58.707511 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2a474-bb72-4355-ae7a-c805c2f34ae4 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:58.716781 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2a474-bb72-4355-ae7a-c805c2f34ae4 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:58.721395 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2292f89-47df-4f60-87d0-d0c2556f9e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:58.721999 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2292f89-47df-4f60-87d0-d0c2556f9e0f admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:58.722283 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2292f89-47df-4f60-87d0-d0c2556f9e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:58.722554 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2292f89-47df-4f60-87d0-d0c2556f9e0f admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:58.723215 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2292f89-47df-4f60-87d0-d0c2556f9e0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:58.725156 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2292f89-47df-4f60-87d0-d0c2556f9e0f admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:58.753425 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1287/2548] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:58:58 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:58:58.757877 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d56e0-5880-48e1-b2db-9d382747394f admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:58.772028 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d56e0-5880-48e1-b2db-9d382747394f admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:58.832015 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c4aa7c-f890-41df-9b16-da9c82cfc510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:58.832626 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4aa7c-f890-41df-9b16-da9c82cfc510 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:58.832908 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4aa7c-f890-41df-9b16-da9c82cfc510 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:58.833184 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4aa7c-f890-41df-9b16-da9c82cfc510 admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:58.833917 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c4aa7c-f890-41df-9b16-da9c82cfc510 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:58.835922 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c4aa7c-f890-41df-9b16-da9c82cfc510 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:58.856502 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1262/2549] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:58:58 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:58:58.860806 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2292f89-47df-4f60-87d0-d0c2556f9e0f admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:58.871709 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2292f89-47df-4f60-87d0-d0c2556f9e0f admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:58.877073 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c5eef-ab19-4410-8d2f-1f43a7619ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:58.882115 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c5eef-ab19-4410-8d2f-1f43a7619ebf admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:58.882115 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c5eef-ab19-4410-8d2f-1f43a7619ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:58.882115 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c5eef-ab19-4410-8d2f-1f43a7619ebf admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:58.882115 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5c5eef-ab19-4410-8d2f-1f43a7619ebf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:58.882115 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5c5eef-ab19-4410-8d2f-1f43a7619ebf admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:58.893011 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1288/2550] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:58:58.897457 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c4aa7c-f890-41df-9b16-da9c82cfc510 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:58.908049 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c4aa7c-f890-41df-9b16-da9c82cfc510 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:58.912812 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd8e3a-6ea4-4343-9b3e-f06ca804235d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:58.913463 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd8e3a-6ea4-4343-9b3e-f06ca804235d admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:58.913746 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd8e3a-6ea4-4343-9b3e-f06ca804235d admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:58.914020 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd8e3a-6ea4-4343-9b3e-f06ca804235d admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:58.914722 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfd8e3a-6ea4-4343-9b3e-f06ca804235d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:58.916497 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfd8e3a-6ea4-4343-9b3e-f06ca804235d admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:58.927252 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1263/2551] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 12:58:58.932274 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c5eef-ab19-4410-8d2f-1f43a7619ebf admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:58.942463 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c5eef-ab19-4410-8d2f-1f43a7619ebf admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:58.947973 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245a473-a466-4c62-9c07-209a8002269c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:58.948626 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245a473-a466-4c62-9c07-209a8002269c admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:58.948905 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245a473-a466-4c62-9c07-209a8002269c admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:58.949176 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245a473-a466-4c62-9c07-209a8002269c admin admin] PATH_INFO: `/v3/projects/d77d2a6745c44c4580330b763921b972/users/c9aef35cc412440eace558b864fc72e1/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:58.963944 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d245a473-a466-4c62-9c07-209a8002269c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d77d2a6745c44c4580330b763921b972, user_id=c9aef35cc412440eace558b864fc72e1, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:58.966653 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d245a473-a466-4c62-9c07-209a8002269c admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:58.979983 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1289/2552] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:58:58 2022] PUT /identity/v3/projects/d77d2a6745c44c4580330b763921b972/users/c9aef35cc412440eace558b864fc72e1/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:58:58.990286 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae67d9-0f10-4b34-b09b-f73fb1646f5d None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:58.990610 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae67d9-0f10-4b34-b09b-f73fb1646f5d None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:58.990890 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae67d9-0f10-4b34-b09b-f73fb1646f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:59.020578 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-f5ae67d9-0f10-4b34-b09b-f73fb1646f5d None None] MFA Rules not processed for user `c9aef35cc412440eace558b864fc72e1`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:58:59.021820 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-f5ae67d9-0f10-4b34-b09b-f73fb1646f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:59.087241 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1264/2553] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:58:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:58:59.091941 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245a473-a466-4c62-9c07-209a8002269c admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:59.153442 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:58:59.161466 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245a473-a466-4c62-9c07-209a8002269c admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:59.168167 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a06f8-3e40-415d-a563-20cf227ceced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:59.168872 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a06f8-3e40-415d-a563-20cf227ceced admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:59.169149 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a06f8-3e40-415d-a563-20cf227ceced admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:59.169426 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a06f8-3e40-415d-a563-20cf227ceced admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:59.170091 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0a06f8-3e40-415d-a563-20cf227ceced admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:59.172068 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0a06f8-3e40-415d-a563-20cf227ceced admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:59.200764 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1290/2554] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:58:59 2022] POST /identity/v3/projects => generated 387 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:58:59.204844 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae67d9-0f10-4b34-b09b-f73fb1646f5d None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:59.215773 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae67d9-0f10-4b34-b09b-f73fb1646f5d None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:59.262546 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:58:59.273277 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaf5127-d6a8-45fb-9c51-d06971d6039a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:59.273906 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf5127-d6a8-45fb-9c51-d06971d6039a admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:59.274199 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf5127-d6a8-45fb-9c51-d06971d6039a admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:59.274473 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf5127-d6a8-45fb-9c51-d06971d6039a admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:59.275141 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaf5127-d6a8-45fb-9c51-d06971d6039a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:59.277095 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaf5127-d6a8-45fb-9c51-d06971d6039a admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:59.298320 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1265/2555] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:58:59 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:58:59.302144 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a06f8-3e40-415d-a563-20cf227ceced admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:59.312904 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a06f8-3e40-415d-a563-20cf227ceced admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:59.317432 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6053950d-e050-4d91-9073-408b21892814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:59.318128 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053950d-e050-4d91-9073-408b21892814 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:59.318410 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053950d-e050-4d91-9073-408b21892814 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:59.318682 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053950d-e050-4d91-9073-408b21892814 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:59.319357 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6053950d-e050-4d91-9073-408b21892814 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:59.321055 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6053950d-e050-4d91-9073-408b21892814 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:59.332266 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1291/2556] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:58:59.336285 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaf5127-d6a8-45fb-9c51-d06971d6039a admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:59.346873 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaf5127-d6a8-45fb-9c51-d06971d6039a admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:59.351505 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3996460-3b17-46de-b95b-0b5db81e6484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:59.352155 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3996460-3b17-46de-b95b-0b5db81e6484 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:59.353047 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3996460-3b17-46de-b95b-0b5db81e6484 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:59.353324 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3996460-3b17-46de-b95b-0b5db81e6484 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:59.354020 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3996460-3b17-46de-b95b-0b5db81e6484 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:59.355626 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3996460-3b17-46de-b95b-0b5db81e6484 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:59.365827 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1266/2557] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:58:59.369569 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6053950d-e050-4d91-9073-408b21892814 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:59.379655 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6053950d-e050-4d91-9073-408b21892814 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:59.384561 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4218c81b-e417-4ae6-b79e-28fea75bc299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:59.385215 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4218c81b-e417-4ae6-b79e-28fea75bc299 admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:59.385498 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4218c81b-e417-4ae6-b79e-28fea75bc299 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:59.385778 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4218c81b-e417-4ae6-b79e-28fea75bc299 admin admin] PATH_INFO: `/v3/projects/7e9b9d9891324cb893a207902e6078df/users/2b48f189ef2e4f53be824b353810caf6/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:59.401102 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4218c81b-e417-4ae6-b79e-28fea75bc299 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e9b9d9891324cb893a207902e6078df, user_id=2b48f189ef2e4f53be824b353810caf6, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:59.404082 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4218c81b-e417-4ae6-b79e-28fea75bc299 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:59.417452 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1292/2558] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:58:59 2022] PUT /identity/v3/projects/7e9b9d9891324cb893a207902e6078df/users/2b48f189ef2e4f53be824b353810caf6/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:58:59.428764 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36f740-ab01-4346-8a80-4bf6d615b662 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:59.429055 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36f740-ab01-4346-8a80-4bf6d615b662 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:59.429331 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36f740-ab01-4346-8a80-4bf6d615b662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:59.457229 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-fc36f740-ab01-4346-8a80-4bf6d615b662 None None] MFA Rules not processed for user `2b48f189ef2e4f53be824b353810caf6`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:58:59.458446 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-fc36f740-ab01-4346-8a80-4bf6d615b662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:59.521771 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1267/2559] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:58:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:58:59.526860 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4218c81b-e417-4ae6-b79e-28fea75bc299 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:59.587302 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4218c81b-e417-4ae6-b79e-28fea75bc299 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:59.591885 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375fe806-fee2-4ea8-b2f1-80059a5b1ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:59.592495 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375fe806-fee2-4ea8-b2f1-80059a5b1ac4 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:59.592774 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375fe806-fee2-4ea8-b2f1-80059a5b1ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:59.593046 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375fe806-fee2-4ea8-b2f1-80059a5b1ac4 admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:59.593702 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375fe806-fee2-4ea8-b2f1-80059a5b1ac4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:59.595671 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375fe806-fee2-4ea8-b2f1-80059a5b1ac4 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:59.622483 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1293/2560] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:58:59 2022] POST /identity/v3/projects => generated 387 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:58:59.626511 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36f740-ab01-4346-8a80-4bf6d615b662 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:59.637710 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36f740-ab01-4346-8a80-4bf6d615b662 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:59.695560 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa4a1c-3f05-4f4a-8dbf-1cd451b6c3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:59.696864 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa4a1c-3f05-4f4a-8dbf-1cd451b6c3a4 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:59.697143 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa4a1c-3f05-4f4a-8dbf-1cd451b6c3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:59.697415 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa4a1c-3f05-4f4a-8dbf-1cd451b6c3a4 admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:59.698090 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fa4a1c-3f05-4f4a-8dbf-1cd451b6c3a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:59.700074 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fa4a1c-3f05-4f4a-8dbf-1cd451b6c3a4 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:59.724071 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1268/2561] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:58:59 2022] POST /identity/v3/users => generated 300 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:58:59.725318 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375fe806-fee2-4ea8-b2f1-80059a5b1ac4 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:59.735908 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375fe806-fee2-4ea8-b2f1-80059a5b1ac4 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:59.741286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010222a7-7352-49ba-ae72-b0ed79ae5622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:59.741907 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010222a7-7352-49ba-ae72-b0ed79ae5622 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:59.742192 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010222a7-7352-49ba-ae72-b0ed79ae5622 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:59.742470 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010222a7-7352-49ba-ae72-b0ed79ae5622 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:59.743201 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010222a7-7352-49ba-ae72-b0ed79ae5622 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:59.744852 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010222a7-7352-49ba-ae72-b0ed79ae5622 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:59.755864 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1294/2562] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:58:59.761220 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa4a1c-3f05-4f4a-8dbf-1cd451b6c3a4 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:59.772156 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa4a1c-3f05-4f4a-8dbf-1cd451b6c3a4 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:59.777996 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9670da-4ea9-47c0-978f-6f62d3c05bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:59.778745 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9670da-4ea9-47c0-978f-6f62d3c05bf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:59.779030 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9670da-4ea9-47c0-978f-6f62d3c05bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:59.779311 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9670da-4ea9-47c0-978f-6f62d3c05bf5 admin admin] PATH_INFO: `/v3/projects/62336cafb4584ad1ab703393c0d51719/users/546e32c181bf4844b5debe7b52265735/roles/0e78444e43c744668178971af4087ca0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:59.795750 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9670da-4ea9-47c0-978f-6f62d3c05bf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62336cafb4584ad1ab703393c0d51719, user_id=546e32c181bf4844b5debe7b52265735, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:59.798549 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9670da-4ea9-47c0-978f-6f62d3c05bf5 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:59.812635 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1269/2563] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:58:59 2022] PUT /identity/v3/projects/62336cafb4584ad1ab703393c0d51719/users/546e32c181bf4844b5debe7b52265735/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:58:59.816811 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010222a7-7352-49ba-ae72-b0ed79ae5622 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:59.828045 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010222a7-7352-49ba-ae72-b0ed79ae5622 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:59.884794 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa15be9-b4fe-4de3-b857-bb263f8f60dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:59.885417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa15be9-b4fe-4de3-b857-bb263f8f60dc admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:59.885702 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa15be9-b4fe-4de3-b857-bb263f8f60dc admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:59.885978 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa15be9-b4fe-4de3-b857-bb263f8f60dc admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:59.886678 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa15be9-b4fe-4de3-b857-bb263f8f60dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:59.888336 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa15be9-b4fe-4de3-b857-bb263f8f60dc admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:59.898986 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1295/2564] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:58:59.905587 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9670da-4ea9-47c0-978f-6f62d3c05bf5 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:58:59.916079 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9670da-4ea9-47c0-978f-6f62d3c05bf5 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:58:59.920746 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c33461-8af7-4436-8510-96f549822b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:58:59.921398 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c33461-8af7-4436-8510-96f549822b31 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:59.921678 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c33461-8af7-4436-8510-96f549822b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:59.921951 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c33461-8af7-4436-8510-96f549822b31 admin admin] PATH_INFO: `/v3/projects/62336cafb4584ad1ab703393c0d51719/users/546e32c181bf4844b5debe7b52265735/roles/0e78444e43c744668178971af4087ca0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:59.923914 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c33461-8af7-4436-8510-96f549822b31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62336cafb4584ad1ab703393c0d51719, user_id=546e32c181bf4844b5debe7b52265735, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:58:59.926579 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c33461-8af7-4436-8510-96f549822b31 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:58:59.937157 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1270/2565] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:58:59 2022] PUT /identity/v3/projects/62336cafb4584ad1ab703393c0d51719/users/546e32c181bf4844b5debe7b52265735/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:58:59.948467 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687741b2-e1f1-4fb4-b19b-a1d108c4179c None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:58:59.948695 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687741b2-e1f1-4fb4-b19b-a1d108c4179c None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:58:59.948977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687741b2-e1f1-4fb4-b19b-a1d108c4179c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:58:59.979663 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-687741b2-e1f1-4fb4-b19b-a1d108c4179c None None] MFA Rules not processed for user `546e32c181bf4844b5debe7b52265735`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:58:59.980905 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-687741b2-e1f1-4fb4-b19b-a1d108c4179c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:58:59.984959 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:59:00.054708 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1296/2566] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:58:59 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:59:00.065918 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c33461-8af7-4436-8510-96f549822b31 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:00.067621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-28c33461-8af7-4436-8510-96f549822b31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:00.082274 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:59:00.128265 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c33461-8af7-4436-8510-96f549822b31 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:00.132239 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82913f10-a756-4bbb-bb2f-6980c3318d9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:00.132849 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82913f10-a756-4bbb-bb2f-6980c3318d9c service cinder] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:00.133128 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82913f10-a756-4bbb-bb2f-6980c3318d9c service cinder] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:00.133426 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82913f10-a756-4bbb-bb2f-6980c3318d9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:00.143964 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82913f10-a756-4bbb-bb2f-6980c3318d9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:00.145408 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82913f10-a756-4bbb-bb2f-6980c3318d9c service cinder] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:00.155548 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1271/2567] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:59:00 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:59:00.685279 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687741b2-e1f1-4fb4-b19b-a1d108c4179c None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:00.695218 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687741b2-e1f1-4fb4-b19b-a1d108c4179c None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:00.698826 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd11b1da-e344-4bf8-ae22-27388e8a16d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:00.699432 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd11b1da-e344-4bf8-ae22-27388e8a16d5 service cinder] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:00.699742 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd11b1da-e344-4bf8-ae22-27388e8a16d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:00.700012 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd11b1da-e344-4bf8-ae22-27388e8a16d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:00.761688 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd11b1da-e344-4bf8-ae22-27388e8a16d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:00.763139 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd11b1da-e344-4bf8-ae22-27388e8a16d5 service cinder] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:00.772684 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1297/2568] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:59:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:59:04.436969 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77984d-f144-4370-a5cb-4be22495b6b7 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:04.437488 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77984d-f144-4370-a5cb-4be22495b6b7 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:04.437488 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77984d-f144-4370-a5cb-4be22495b6b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:04.456701 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-0d77984d-f144-4370-a5cb-4be22495b6b7 None None] MFA Rules not processed for user `8e8b5c62a961409baaaa101f61437f1e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:59:04.457911 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-0d77984d-f144-4370-a5cb-4be22495b6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:04.512882 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1272/2569] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:59:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:59:04.862226 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd11b1da-e344-4bf8-ae22-27388e8a16d5 service cinder] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:04.863718 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-bd11b1da-e344-4bf8-ae22-27388e8a16d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:04.875330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd11b1da-e344-4bf8-ae22-27388e8a16d5 service cinder] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:04.878973 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0ff6ef-f126-4f36-837f-989a4df2811f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:04.879583 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0ff6ef-f126-4f36-837f-989a4df2811f service cinder] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:04.879908 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0ff6ef-f126-4f36-837f-989a4df2811f service cinder] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:04.880185 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0ff6ef-f126-4f36-837f-989a4df2811f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:04.946556 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0ff6ef-f126-4f36-837f-989a4df2811f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:04.948103 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0ff6ef-f126-4f36-837f-989a4df2811f service cinder] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:04.957172 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1298/2570] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:59:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:59:07.144241 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77984d-f144-4370-a5cb-4be22495b6b7 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:07.224862 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77984d-f144-4370-a5cb-4be22495b6b7 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:07.230016 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa3f8f-1f9b-4a02-89ec-bccb61c4e506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:07.230658 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa3f8f-1f9b-4a02-89ec-bccb61c4e506 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:07.230939 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa3f8f-1f9b-4a02-89ec-bccb61c4e506 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:07.231216 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa3f8f-1f9b-4a02-89ec-bccb61c4e506 admin admin] PATH_INFO: `/v3/users/c9aef35cc412440eace558b864fc72e1` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:07.232415 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aa3f8f-1f9b-4a02-89ec-bccb61c4e506 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9aef35cc412440eace558b864fc72e1)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:07.234386 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aa3f8f-1f9b-4a02-89ec-bccb61c4e506 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:07.286735 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-a4aa3f8f-1f9b-4a02-89ec-bccb61c4e506 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9aef35cc412440eace558b864fc72e1'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:07.288013 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-a4aa3f8f-1f9b-4a02-89ec-bccb61c4e506 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9aef35cc412440eace558b864fc72e1'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:07.289139 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-a4aa3f8f-1f9b-4a02-89ec-bccb61c4e506 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9aef35cc412440eace558b864fc72e1'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:07.303933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:59:07.308187 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-a4aa3f8f-1f9b-4a02-89ec-bccb61c4e506 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9aef35cc412440eace558b864fc72e1'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:07.319603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-a4aa3f8f-1f9b-4a02-89ec-bccb61c4e506 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9aef35cc412440eace558b864fc72e1'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:07.340765 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1273/2571] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:59:07 2022] DELETE /identity/v3/users/c9aef35cc412440eace558b864fc72e1 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:59:07.345304 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f2acf0-a3f6-4517-8431-bade43e11c2d None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:07.345899 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f2acf0-a3f6-4517-8431-bade43e11c2d None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:07.345985 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f2acf0-a3f6-4517-8431-bade43e11c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:07.374687 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-c9f2acf0-a3f6-4517-8431-bade43e11c2d None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:59:07.378466 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-c9f2acf0-a3f6-4517-8431-bade43e11c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:07.463074 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:59:07.505033 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1299/2572] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:59:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:59:07.520925 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa3f8f-1f9b-4a02-89ec-bccb61c4e506 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:07.522481 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-a4aa3f8f-1f9b-4a02-89ec-bccb61c4e506 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:07.584302 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa3f8f-1f9b-4a02-89ec-bccb61c4e506 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:07.591173 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49fc6d-c9f9-4e86-9522-9ee77756a2d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:07.591383 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49fc6d-c9f9-4e86-9522-9ee77756a2d1 service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:07.591605 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49fc6d-c9f9-4e86-9522-9ee77756a2d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:07.592112 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49fc6d-c9f9-4e86-9522-9ee77756a2d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:07.603276 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d49fc6d-c9f9-4e86-9522-9ee77756a2d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:07.604940 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d49fc6d-c9f9-4e86-9522-9ee77756a2d1 service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:07.615747 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1274/2573] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:59:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:59:08.064388 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f2acf0-a3f6-4517-8431-bade43e11c2d None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:08.066074 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-c9f2acf0-a3f6-4517-8431-bade43e11c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:08.086014 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f2acf0-a3f6-4517-8431-bade43e11c2d None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:08.092642 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df982d9-3c9b-43de-aa7f-c71ea000273b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:08.093278 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df982d9-3c9b-43de-aa7f-c71ea000273b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:08.093562 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df982d9-3c9b-43de-aa7f-c71ea000273b admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:08.093838 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df982d9-3c9b-43de-aa7f-c71ea000273b admin admin] PATH_INFO: `/v3/projects/d77d2a6745c44c4580330b763921b972` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:08.095012 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df982d9-3c9b-43de-aa7f-c71ea000273b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d77d2a6745c44c4580330b763921b972)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:08.096980 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df982d9-3c9b-43de-aa7f-c71ea000273b admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:08.148795 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-3df982d9-3c9b-43de-aa7f-c71ea000273b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd77d2a6745c44c4580330b763921b972', 'request_id': 'req-3df982d9-3c9b-43de-aa7f-c71ea000273b'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:08.155554 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-3df982d9-3c9b-43de-aa7f-c71ea000273b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd77d2a6745c44c4580330b763921b972', 'request_id': 'req-3df982d9-3c9b-43de-aa7f-c71ea000273b'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:08.163493 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-3df982d9-3c9b-43de-aa7f-c71ea000273b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd77d2a6745c44c4580330b763921b972', 'request_id': 'req-3df982d9-3c9b-43de-aa7f-c71ea000273b'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:08.174276 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-3df982d9-3c9b-43de-aa7f-c71ea000273b admin admin] The token cache is being invalidate because project d77d2a6745c44c4580330b763921b972 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:59:08.174559 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-3df982d9-3c9b-43de-aa7f-c71ea000273b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:08.175242 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-3df982d9-3c9b-43de-aa7f-c71ea000273b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:08.176873 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1300/2574] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:59:08 2022] DELETE /identity/v3/projects/d77d2a6745c44c4580330b763921b972 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:59:08.180962 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49fc6d-c9f9-4e86-9522-9ee77756a2d1 service neutron] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:08.191058 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49fc6d-c9f9-4e86-9522-9ee77756a2d1 service neutron] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:08.249099 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36322ba-e554-4e4a-80c2-1a03010143ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:08.249728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36322ba-e554-4e4a-80c2-1a03010143ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:08.250013 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36322ba-e554-4e4a-80c2-1a03010143ec admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:08.250290 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36322ba-e554-4e4a-80c2-1a03010143ec admin admin] PATH_INFO: `/v3/users/2b48f189ef2e4f53be824b353810caf6` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:08.251444 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36322ba-e554-4e4a-80c2-1a03010143ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b48f189ef2e4f53be824b353810caf6)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:08.253407 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36322ba-e554-4e4a-80c2-1a03010143ec admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:08.299845 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-a36322ba-e554-4e4a-80c2-1a03010143ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b48f189ef2e4f53be824b353810caf6'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:08.300504 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-a36322ba-e554-4e4a-80c2-1a03010143ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b48f189ef2e4f53be824b353810caf6'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:08.301177 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-a36322ba-e554-4e4a-80c2-1a03010143ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b48f189ef2e4f53be824b353810caf6'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:08.312992 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-a36322ba-e554-4e4a-80c2-1a03010143ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b48f189ef2e4f53be824b353810caf6'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:08.319659 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-a36322ba-e554-4e4a-80c2-1a03010143ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b48f189ef2e4f53be824b353810caf6'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:08.329047 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:59:08.334209 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1275/2575] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:59:08 2022] DELETE /identity/v3/users/2b48f189ef2e4f53be824b353810caf6 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:59:08.713875 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df982d9-3c9b-43de-aa7f-c71ea000273b admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:08.716546 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-3df982d9-3c9b-43de-aa7f-c71ea000273b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:08.741178 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df982d9-3c9b-43de-aa7f-c71ea000273b admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:08.800120 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f51a37-50ed-4b21-abc6-49020c537065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:08.800883 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f51a37-50ed-4b21-abc6-49020c537065 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:08.801169 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f51a37-50ed-4b21-abc6-49020c537065 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:08.801445 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f51a37-50ed-4b21-abc6-49020c537065 admin admin] PATH_INFO: `/v3/projects/7e9b9d9891324cb893a207902e6078df` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:08.802716 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f51a37-50ed-4b21-abc6-49020c537065 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e9b9d9891324cb893a207902e6078df)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:08.804714 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f51a37-50ed-4b21-abc6-49020c537065 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:08.864023 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-20f51a37-50ed-4b21-abc6-49020c537065 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e9b9d9891324cb893a207902e6078df', 'request_id': 'req-20f51a37-50ed-4b21-abc6-49020c537065'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:08.870740 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-20f51a37-50ed-4b21-abc6-49020c537065 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e9b9d9891324cb893a207902e6078df', 'request_id': 'req-20f51a37-50ed-4b21-abc6-49020c537065'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:08.879672 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-20f51a37-50ed-4b21-abc6-49020c537065 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e9b9d9891324cb893a207902e6078df', 'request_id': 'req-20f51a37-50ed-4b21-abc6-49020c537065'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:08.882476 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:59:08.893822 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-20f51a37-50ed-4b21-abc6-49020c537065 admin admin] The token cache is being invalidate because project 7e9b9d9891324cb893a207902e6078df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:59:08.894383 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-20f51a37-50ed-4b21-abc6-49020c537065 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:08.895752 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-20f51a37-50ed-4b21-abc6-49020c537065 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:08.897456 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1301/2576] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:59:08 2022] DELETE /identity/v3/projects/7e9b9d9891324cb893a207902e6078df => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:59:08.902525 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36322ba-e554-4e4a-80c2-1a03010143ec admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:08.904165 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-a36322ba-e554-4e4a-80c2-1a03010143ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:08.917535 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36322ba-e554-4e4a-80c2-1a03010143ec admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:08.995153 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37509bb5-07bf-4b49-a585-79a89c608c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:08.996301 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37509bb5-07bf-4b49-a585-79a89c608c28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:08.996855 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37509bb5-07bf-4b49-a585-79a89c608c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:08.997367 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37509bb5-07bf-4b49-a585-79a89c608c28 admin admin] PATH_INFO: `/v3/users/546e32c181bf4844b5debe7b52265735` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:08.999421 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37509bb5-07bf-4b49-a585-79a89c608c28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=546e32c181bf4844b5debe7b52265735)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:09.003302 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37509bb5-07bf-4b49-a585-79a89c608c28 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:09.064424 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-37509bb5-07bf-4b49-a585-79a89c608c28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '546e32c181bf4844b5debe7b52265735'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:09.065335 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-37509bb5-07bf-4b49-a585-79a89c608c28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '546e32c181bf4844b5debe7b52265735'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:09.065996 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-37509bb5-07bf-4b49-a585-79a89c608c28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '546e32c181bf4844b5debe7b52265735'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:09.079434 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-37509bb5-07bf-4b49-a585-79a89c608c28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '546e32c181bf4844b5debe7b52265735'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:09.088293 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-37509bb5-07bf-4b49-a585-79a89c608c28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '546e32c181bf4844b5debe7b52265735'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:09.104032 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1276/2577] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:59:08 2022] DELETE /identity/v3/users/546e32c181bf4844b5debe7b52265735 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:59:09.451886 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f51a37-50ed-4b21-abc6-49020c537065 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:09.453483 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-20f51a37-50ed-4b21-abc6-49020c537065 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:09.467510 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f51a37-50ed-4b21-abc6-49020c537065 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:09.531472 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aab655-9de7-48f4-83d3-1b72c28e7923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:09.532132 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aab655-9de7-48f4-83d3-1b72c28e7923 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:09.532411 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aab655-9de7-48f4-83d3-1b72c28e7923 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:09.532711 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aab655-9de7-48f4-83d3-1b72c28e7923 admin admin] PATH_INFO: `/v3/projects/62336cafb4584ad1ab703393c0d51719` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:09.533855 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aab655-9de7-48f4-83d3-1b72c28e7923 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62336cafb4584ad1ab703393c0d51719)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:09.535793 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aab655-9de7-48f4-83d3-1b72c28e7923 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:09.583760 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-66aab655-9de7-48f4-83d3-1b72c28e7923 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62336cafb4584ad1ab703393c0d51719', 'request_id': 'req-66aab655-9de7-48f4-83d3-1b72c28e7923'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:09.589969 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-66aab655-9de7-48f4-83d3-1b72c28e7923 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62336cafb4584ad1ab703393c0d51719', 'request_id': 'req-66aab655-9de7-48f4-83d3-1b72c28e7923'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:09.597965 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-66aab655-9de7-48f4-83d3-1b72c28e7923 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62336cafb4584ad1ab703393c0d51719', 'request_id': 'req-66aab655-9de7-48f4-83d3-1b72c28e7923'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:09.610920 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-66aab655-9de7-48f4-83d3-1b72c28e7923 admin admin] The token cache is being invalidate because project 62336cafb4584ad1ab703393c0d51719 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:59:09.611326 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-66aab655-9de7-48f4-83d3-1b72c28e7923 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:09.612322 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-66aab655-9de7-48f4-83d3-1b72c28e7923 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:09.614096 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1302/2578] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:59:09 2022] DELETE /identity/v3/projects/62336cafb4584ad1ab703393c0d51719 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:59:09.620531 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f140cd9-9aae-4ff3-af01-4e066818847b None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:09.621044 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f140cd9-9aae-4ff3-af01-4e066818847b None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:09.621145 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f140cd9-9aae-4ff3-af01-4e066818847b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:09.641794 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-5f140cd9-9aae-4ff3-af01-4e066818847b None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:59:09.643001 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-5f140cd9-9aae-4ff3-af01-4e066818847b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:09.702236 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1277/2579] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:59:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:59:09.710856 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4781666-7d7b-4322-b678-9249e8f4f057 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:09.711148 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4781666-7d7b-4322-b678-9249e8f4f057 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:09.711468 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4781666-7d7b-4322-b678-9249e8f4f057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:09.731514 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-a4781666-7d7b-4322-b678-9249e8f4f057 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:59:09.732743 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-a4781666-7d7b-4322-b678-9249e8f4f057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:09.740778 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1303/2580] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:59:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:59:09.745789 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f140cd9-9aae-4ff3-af01-4e066818847b None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:09.757984 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f140cd9-9aae-4ff3-af01-4e066818847b None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:09.763674 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00875839-a032-4b1a-821a-673c1feb06ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:09.763939 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00875839-a032-4b1a-821a-673c1feb06ec admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:09.771296 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00875839-a032-4b1a-821a-673c1feb06ec admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:09.771296 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00875839-a032-4b1a-821a-673c1feb06ec admin admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:09.771296 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00875839-a032-4b1a-821a-673c1feb06ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:59:09.771296 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00875839-a032-4b1a-821a-673c1feb06ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:09.771296 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00875839-a032-4b1a-821a-673c1feb06ec admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:09.788130 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1278/2581] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:59:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:59:09.789444 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4781666-7d7b-4322-b678-9249e8f4f057 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:09.800175 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4781666-7d7b-4322-b678-9249e8f4f057 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:09.809965 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a113967-57fa-4a71-b8b8-cd8b68c62cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:09.810911 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a113967-57fa-4a71-b8b8-cd8b68c62cca admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:09.811855 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a113967-57fa-4a71-b8b8-cd8b68c62cca admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:09.812310 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a113967-57fa-4a71-b8b8-cd8b68c62cca admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:09.813374 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a113967-57fa-4a71-b8b8-cd8b68c62cca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:09.816318 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a113967-57fa-4a71-b8b8-cd8b68c62cca admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:09.858589 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1304/2582] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:59:09 2022] POST /identity/v3/projects => generated 391 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:59:09.862892 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00875839-a032-4b1a-821a-673c1feb06ec admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:09.874003 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00875839-a032-4b1a-821a-673c1feb06ec admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:09.878776 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:59:09.957222 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d6848-741b-4c63-b54d-bd0a7d93fc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:09.957866 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d6848-741b-4c63-b54d-bd0a7d93fc3b admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:09.958415 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d6848-741b-4c63-b54d-bd0a7d93fc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:09.958754 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d6848-741b-4c63-b54d-bd0a7d93fc3b admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:09.959501 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d6848-741b-4c63-b54d-bd0a7d93fc3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:09.962070 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d6848-741b-4c63-b54d-bd0a7d93fc3b admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:09.990163 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1279/2583] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:59:09 2022] POST /identity/v3/users => generated 296 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:59:09.991671 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a113967-57fa-4a71-b8b8-cd8b68c62cca admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:10.002361 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a113967-57fa-4a71-b8b8-cd8b68c62cca admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:10.007477 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2ba97-adf8-457c-9e41-46a7dabab963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:10.008208 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2ba97-adf8-457c-9e41-46a7dabab963 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:10.008601 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2ba97-adf8-457c-9e41-46a7dabab963 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:10.008877 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2ba97-adf8-457c-9e41-46a7dabab963 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:10.009586 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f2ba97-adf8-457c-9e41-46a7dabab963 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:10.011228 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f2ba97-adf8-457c-9e41-46a7dabab963 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:10.022088 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1305/2584] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:59:10.026624 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d6848-741b-4c63-b54d-bd0a7d93fc3b admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:10.037432 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d6848-741b-4c63-b54d-bd0a7d93fc3b admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:10.042684 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e7fccd-96ca-473c-94cf-7cb8cad33848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:10.043592 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e7fccd-96ca-473c-94cf-7cb8cad33848 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:10.044009 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e7fccd-96ca-473c-94cf-7cb8cad33848 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:10.044461 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e7fccd-96ca-473c-94cf-7cb8cad33848 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:10.045326 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e7fccd-96ca-473c-94cf-7cb8cad33848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:10.047363 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e7fccd-96ca-473c-94cf-7cb8cad33848 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:10.060811 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1280/2585] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:59:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 12:59:10.065117 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2ba97-adf8-457c-9e41-46a7dabab963 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:10.075924 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2ba97-adf8-457c-9e41-46a7dabab963 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:10.080640 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff038d-fad2-49e4-bcc0-2c0bbbed7f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:10.081299 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff038d-fad2-49e4-bcc0-2c0bbbed7f3c admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:10.081579 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff038d-fad2-49e4-bcc0-2c0bbbed7f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:10.081851 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff038d-fad2-49e4-bcc0-2c0bbbed7f3c admin admin] PATH_INFO: `/v3/projects/4b02dde2d0ba4aa3a4cc064d41601297/users/0bc1d77173674519a25c14ffffdd78b3/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:10.098599 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ff038d-fad2-49e4-bcc0-2c0bbbed7f3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b02dde2d0ba4aa3a4cc064d41601297, user_id=0bc1d77173674519a25c14ffffdd78b3, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:10.101346 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ff038d-fad2-49e4-bcc0-2c0bbbed7f3c admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:10.114732 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1306/2586] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:59:10 2022] PUT /identity/v3/projects/4b02dde2d0ba4aa3a4cc064d41601297/users/0bc1d77173674519a25c14ffffdd78b3/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:59:10.127624 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac51cb-790b-4232-89c6-ce4d50492be9 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:10.127846 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac51cb-790b-4232-89c6-ce4d50492be9 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:10.128135 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac51cb-790b-4232-89c6-ce4d50492be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:10.157156 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-ceac51cb-790b-4232-89c6-ce4d50492be9 None None] MFA Rules not processed for user `0bc1d77173674519a25c14ffffdd78b3`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:59:10.158399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-ceac51cb-790b-4232-89c6-ce4d50492be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:10.226135 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1281/2587] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:59:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:59:10.231280 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff038d-fad2-49e4-bcc0-2c0bbbed7f3c admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:10.250979 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:59:10.435778 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff038d-fad2-49e4-bcc0-2c0bbbed7f3c admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:10.440423 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3261cab-c435-4ef6-9e33-5029a99ea112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:10.441051 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3261cab-c435-4ef6-9e33-5029a99ea112 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:10.441334 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3261cab-c435-4ef6-9e33-5029a99ea112 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:10.441639 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3261cab-c435-4ef6-9e33-5029a99ea112 admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:10.442306 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3261cab-c435-4ef6-9e33-5029a99ea112 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:10.444257 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3261cab-c435-4ef6-9e33-5029a99ea112 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:10.473958 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1307/2588] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:59:10 2022] POST /identity/v3/projects => generated 391 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:59:10.478963 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac51cb-790b-4232-89c6-ce4d50492be9 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:10.489991 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac51cb-790b-4232-89c6-ce4d50492be9 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:10.547805 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd2c55-7dc9-4aba-96ac-a7640e6c206b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:10.548422 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd2c55-7dc9-4aba-96ac-a7640e6c206b admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:10.548700 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd2c55-7dc9-4aba-96ac-a7640e6c206b admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:10.548969 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd2c55-7dc9-4aba-96ac-a7640e6c206b admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:10.549640 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd2c55-7dc9-4aba-96ac-a7640e6c206b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:10.551574 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd2c55-7dc9-4aba-96ac-a7640e6c206b admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:10.573084 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1282/2589] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:59:10 2022] POST /identity/v3/users => generated 302 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:59:10.577643 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3261cab-c435-4ef6-9e33-5029a99ea112 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:10.588250 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3261cab-c435-4ef6-9e33-5029a99ea112 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:10.592780 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e074ab-70f9-438c-9d69-52648c1eb3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:10.593384 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e074ab-70f9-438c-9d69-52648c1eb3a4 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:10.593661 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e074ab-70f9-438c-9d69-52648c1eb3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:10.593927 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e074ab-70f9-438c-9d69-52648c1eb3a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:10.594610 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e074ab-70f9-438c-9d69-52648c1eb3a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:10.596232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e074ab-70f9-438c-9d69-52648c1eb3a4 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:10.607439 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1308/2590] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:59:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 12:59:10.611621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd2c55-7dc9-4aba-96ac-a7640e6c206b admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:10.618274 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:59:10.622446 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd2c55-7dc9-4aba-96ac-a7640e6c206b admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:10.628056 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17abde34-ff75-4ed1-a1b3-b0997e2a32c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:10.628721 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17abde34-ff75-4ed1-a1b3-b0997e2a32c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:10.628996 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17abde34-ff75-4ed1-a1b3-b0997e2a32c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:10.629268 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17abde34-ff75-4ed1-a1b3-b0997e2a32c0 admin admin] PATH_INFO: `/v3/projects/996c709013264a4ebe52632b0ab1c20c/users/fedf02bdc4664af39aa1a14721e39b26/roles/0e78444e43c744668178971af4087ca0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:10.644620 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17abde34-ff75-4ed1-a1b3-b0997e2a32c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=996c709013264a4ebe52632b0ab1c20c, user_id=fedf02bdc4664af39aa1a14721e39b26, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:10.647262 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17abde34-ff75-4ed1-a1b3-b0997e2a32c0 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:10.661336 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1283/2591] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:59:10 2022] PUT /identity/v3/projects/996c709013264a4ebe52632b0ab1c20c/users/fedf02bdc4664af39aa1a14721e39b26/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:59:10.665968 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e074ab-70f9-438c-9d69-52648c1eb3a4 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:10.676565 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e074ab-70f9-438c-9d69-52648c1eb3a4 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:10.737578 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71315129-ee99-4f5b-b68d-c0a16746dcde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:10.738353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71315129-ee99-4f5b-b68d-c0a16746dcde admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:10.738470 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71315129-ee99-4f5b-b68d-c0a16746dcde admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:10.738710 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71315129-ee99-4f5b-b68d-c0a16746dcde admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:10.739437 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71315129-ee99-4f5b-b68d-c0a16746dcde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:10.741062 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71315129-ee99-4f5b-b68d-c0a16746dcde admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:10.751479 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1309/2592] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:59:10 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 12:59:10.755704 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17abde34-ff75-4ed1-a1b3-b0997e2a32c0 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:10.766275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17abde34-ff75-4ed1-a1b3-b0997e2a32c0 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:10.771152 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090900e6-8d68-404c-a83a-8207cdd21d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:10.771843 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090900e6-8d68-404c-a83a-8207cdd21d6e admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:10.772121 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090900e6-8d68-404c-a83a-8207cdd21d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:10.772390 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090900e6-8d68-404c-a83a-8207cdd21d6e admin admin] PATH_INFO: `/v3/projects/996c709013264a4ebe52632b0ab1c20c/users/fedf02bdc4664af39aa1a14721e39b26/roles/0e78444e43c744668178971af4087ca0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:10.774844 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090900e6-8d68-404c-a83a-8207cdd21d6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=996c709013264a4ebe52632b0ab1c20c, user_id=fedf02bdc4664af39aa1a14721e39b26, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:10.777532 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090900e6-8d68-404c-a83a-8207cdd21d6e admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:10.788220 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1284/2593] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:59:10 2022] PUT /identity/v3/projects/996c709013264a4ebe52632b0ab1c20c/users/fedf02bdc4664af39aa1a14721e39b26/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:59:10.799125 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f4016b-3e6e-4beb-86ab-138d55874e8d None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:10.799420 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f4016b-3e6e-4beb-86ab-138d55874e8d None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:10.799719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f4016b-3e6e-4beb-86ab-138d55874e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:10.829101 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-01f4016b-3e6e-4beb-86ab-138d55874e8d None None] MFA Rules not processed for user `fedf02bdc4664af39aa1a14721e39b26`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:59:10.830389 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-01f4016b-3e6e-4beb-86ab-138d55874e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:10.902527 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1310/2594] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:59:10 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:59:10.914577 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090900e6-8d68-404c-a83a-8207cdd21d6e admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:10.916892 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-090900e6-8d68-404c-a83a-8207cdd21d6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:10.983729 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090900e6-8d68-404c-a83a-8207cdd21d6e admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:10.992806 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a2cf5-faa1-4f67-b42a-4d88855ede6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:10.993467 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a2cf5-faa1-4f67-b42a-4d88855ede6e service cinder] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:10.993759 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a2cf5-faa1-4f67-b42a-4d88855ede6e service cinder] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:10.994030 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a2cf5-faa1-4f67-b42a-4d88855ede6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:11.005698 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8a2cf5-faa1-4f67-b42a-4d88855ede6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:11.007429 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8a2cf5-faa1-4f67-b42a-4d88855ede6e service cinder] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:11.020076 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1285/2595] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:59:10 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:59:11.073979 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f4016b-3e6e-4beb-86ab-138d55874e8d None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:11.075557 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-01f4016b-3e6e-4beb-86ab-138d55874e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:11.089030 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f4016b-3e6e-4beb-86ab-138d55874e8d None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:11.092757 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7493b388-28dd-459d-a654-4f85e624f31e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:11.093374 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493b388-28dd-459d-a654-4f85e624f31e service cinder] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:11.093656 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493b388-28dd-459d-a654-4f85e624f31e service cinder] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:11.093927 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493b388-28dd-459d-a654-4f85e624f31e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:11.104620 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:59:11.158235 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7493b388-28dd-459d-a654-4f85e624f31e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:11.159693 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7493b388-28dd-459d-a654-4f85e624f31e service cinder] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:11.168858 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1311/2596] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:59:11 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:59:14.508965 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a2cf5-faa1-4f67-b42a-4d88855ede6e service cinder] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:14.686658 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a2cf5-faa1-4f67-b42a-4d88855ede6e service cinder] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:14.694278 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e9a6df-dce2-4c3c-9742-b4e51929ad71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:14.694934 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e9a6df-dce2-4c3c-9742-b4e51929ad71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:14.695213 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e9a6df-dce2-4c3c-9742-b4e51929ad71 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:14.695487 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e9a6df-dce2-4c3c-9742-b4e51929ad71 admin admin] PATH_INFO: `/v3/users/0bc1d77173674519a25c14ffffdd78b3` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:14.697871 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e9a6df-dce2-4c3c-9742-b4e51929ad71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bc1d77173674519a25c14ffffdd78b3)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:14.700262 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e9a6df-dce2-4c3c-9742-b4e51929ad71 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:14.759717 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:59:14.779253 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-23e9a6df-dce2-4c3c-9742-b4e51929ad71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bc1d77173674519a25c14ffffdd78b3'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:14.780681 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-23e9a6df-dce2-4c3c-9742-b4e51929ad71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bc1d77173674519a25c14ffffdd78b3'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:14.782158 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-23e9a6df-dce2-4c3c-9742-b4e51929ad71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bc1d77173674519a25c14ffffdd78b3'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:14.799016 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-23e9a6df-dce2-4c3c-9742-b4e51929ad71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bc1d77173674519a25c14ffffdd78b3'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:14.806643 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-23e9a6df-dce2-4c3c-9742-b4e51929ad71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bc1d77173674519a25c14ffffdd78b3'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:14.825617 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1286/2597] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:59:14 2022] DELETE /identity/v3/users/0bc1d77173674519a25c14ffffdd78b3 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:59:14.830409 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010bb3e2-4a6d-40f3-aa70-132bafe687fe None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:14.830908 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010bb3e2-4a6d-40f3-aa70-132bafe687fe None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:14.831146 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010bb3e2-4a6d-40f3-aa70-132bafe687fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:14.893228 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-010bb3e2-4a6d-40f3-aa70-132bafe687fe None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:59:14.894724 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-010bb3e2-4a6d-40f3-aa70-132bafe687fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:14.964790 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1312/2598] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:59:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:59:14.975514 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e9a6df-dce2-4c3c-9742-b4e51929ad71 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:14.977135 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-23e9a6df-dce2-4c3c-9742-b4e51929ad71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:15.064085 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e9a6df-dce2-4c3c-9742-b4e51929ad71 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:15.068423 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef42842b-1a9c-481f-a287-4c474de81b47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:15.069144 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef42842b-1a9c-481f-a287-4c474de81b47 service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:15.069403 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef42842b-1a9c-481f-a287-4c474de81b47 service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:15.069696 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef42842b-1a9c-481f-a287-4c474de81b47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:15.080487 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef42842b-1a9c-481f-a287-4c474de81b47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:15.081967 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef42842b-1a9c-481f-a287-4c474de81b47 service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:15.093209 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1287/2599] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:59:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:59:15.394072 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010bb3e2-4a6d-40f3-aa70-132bafe687fe None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:15.396002 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-010bb3e2-4a6d-40f3-aa70-132bafe687fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:15.416825 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010bb3e2-4a6d-40f3-aa70-132bafe687fe None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:15.423143 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598239f4-fcf7-4802-bf4f-faab6ca816e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:15.424046 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598239f4-fcf7-4802-bf4f-faab6ca816e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:15.424502 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598239f4-fcf7-4802-bf4f-faab6ca816e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:15.424932 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598239f4-fcf7-4802-bf4f-faab6ca816e0 admin admin] PATH_INFO: `/v3/projects/4b02dde2d0ba4aa3a4cc064d41601297` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:15.426725 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598239f4-fcf7-4802-bf4f-faab6ca816e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b02dde2d0ba4aa3a4cc064d41601297)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:15.429316 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598239f4-fcf7-4802-bf4f-faab6ca816e0 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:15.483984 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-598239f4-fcf7-4802-bf4f-faab6ca816e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b02dde2d0ba4aa3a4cc064d41601297', 'request_id': 'req-598239f4-fcf7-4802-bf4f-faab6ca816e0'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:15.490479 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-598239f4-fcf7-4802-bf4f-faab6ca816e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b02dde2d0ba4aa3a4cc064d41601297', 'request_id': 'req-598239f4-fcf7-4802-bf4f-faab6ca816e0'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:15.498252 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-598239f4-fcf7-4802-bf4f-faab6ca816e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b02dde2d0ba4aa3a4cc064d41601297', 'request_id': 'req-598239f4-fcf7-4802-bf4f-faab6ca816e0'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:15.510353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-598239f4-fcf7-4802-bf4f-faab6ca816e0 admin admin] The token cache is being invalidate because project 4b02dde2d0ba4aa3a4cc064d41601297 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:59:15.510646 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-598239f4-fcf7-4802-bf4f-faab6ca816e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:15.511605 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-598239f4-fcf7-4802-bf4f-faab6ca816e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:15.513235 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1313/2600] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:59:15 2022] DELETE /identity/v3/projects/4b02dde2d0ba4aa3a4cc064d41601297 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:59:15.517563 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef42842b-1a9c-481f-a287-4c474de81b47 service neutron] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:15.528217 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef42842b-1a9c-481f-a287-4c474de81b47 service neutron] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:15.587858 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b205c142-bb70-4539-8825-151d152c087b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:15.588612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205c142-bb70-4539-8825-151d152c087b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:15.589055 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205c142-bb70-4539-8825-151d152c087b admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:15.589239 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205c142-bb70-4539-8825-151d152c087b admin admin] PATH_INFO: `/v3/users/fedf02bdc4664af39aa1a14721e39b26` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:15.590429 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b205c142-bb70-4539-8825-151d152c087b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fedf02bdc4664af39aa1a14721e39b26)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:15.592430 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b205c142-bb70-4539-8825-151d152c087b admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:15.641829 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:59:15.652873 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-b205c142-bb70-4539-8825-151d152c087b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fedf02bdc4664af39aa1a14721e39b26'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:15.653693 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-b205c142-bb70-4539-8825-151d152c087b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fedf02bdc4664af39aa1a14721e39b26'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:15.654768 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-b205c142-bb70-4539-8825-151d152c087b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fedf02bdc4664af39aa1a14721e39b26'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:15.668103 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-b205c142-bb70-4539-8825-151d152c087b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fedf02bdc4664af39aa1a14721e39b26'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:15.675146 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-b205c142-bb70-4539-8825-151d152c087b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fedf02bdc4664af39aa1a14721e39b26'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:15.690683 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1288/2601] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:59:15 2022] DELETE /identity/v3/users/fedf02bdc4664af39aa1a14721e39b26 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:59:16.003795 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598239f4-fcf7-4802-bf4f-faab6ca816e0 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:16.005489 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-598239f4-fcf7-4802-bf4f-faab6ca816e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:16.022144 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598239f4-fcf7-4802-bf4f-faab6ca816e0 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:16.027500 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:59:16.095033 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e8aa5-1e49-43e9-baa2-6e531958c5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:16.095911 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e8aa5-1e49-43e9-baa2-6e531958c5d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:16.096259 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e8aa5-1e49-43e9-baa2-6e531958c5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:16.096599 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e8aa5-1e49-43e9-baa2-6e531958c5d4 admin admin] PATH_INFO: `/v3/projects/996c709013264a4ebe52632b0ab1c20c` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:16.097926 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e8aa5-1e49-43e9-baa2-6e531958c5d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=996c709013264a4ebe52632b0ab1c20c)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:16.099997 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e8aa5-1e49-43e9-baa2-6e531958c5d4 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:16.151872 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-333e8aa5-1e49-43e9-baa2-6e531958c5d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '996c709013264a4ebe52632b0ab1c20c', 'request_id': 'req-333e8aa5-1e49-43e9-baa2-6e531958c5d4'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:16.158009 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-333e8aa5-1e49-43e9-baa2-6e531958c5d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '996c709013264a4ebe52632b0ab1c20c', 'request_id': 'req-333e8aa5-1e49-43e9-baa2-6e531958c5d4'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:16.170268 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-333e8aa5-1e49-43e9-baa2-6e531958c5d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '996c709013264a4ebe52632b0ab1c20c', 'request_id': 'req-333e8aa5-1e49-43e9-baa2-6e531958c5d4'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:16.182965 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-333e8aa5-1e49-43e9-baa2-6e531958c5d4 admin admin] The token cache is being invalidate because project 996c709013264a4ebe52632b0ab1c20c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:59:16.183256 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-333e8aa5-1e49-43e9-baa2-6e531958c5d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:16.183938 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-333e8aa5-1e49-43e9-baa2-6e531958c5d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:16.185515 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1314/2602] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:59:16 2022] DELETE /identity/v3/projects/996c709013264a4ebe52632b0ab1c20c => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:59:16.190348 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2882a8-2784-46f4-99fe-cd8a75cfb6e5 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:16.190645 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2882a8-2784-46f4-99fe-cd8a75cfb6e5 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:16.190928 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2882a8-2784-46f4-99fe-cd8a75cfb6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:16.240828 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-4c2882a8-2784-46f4-99fe-cd8a75cfb6e5 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:59:16.242055 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-4c2882a8-2784-46f4-99fe-cd8a75cfb6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:16.312689 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1289/2603] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:59:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:59:16.377510 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8de00a-c9e6-4cef-ad06-309db6685106 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:16.377796 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8de00a-c9e6-4cef-ad06-309db6685106 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:16.378075 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8de00a-c9e6-4cef-ad06-309db6685106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:16.398862 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-3d8de00a-c9e6-4cef-ad06-309db6685106 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:59:16.400104 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-3d8de00a-c9e6-4cef-ad06-309db6685106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:16.408961 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1315/2604] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:59:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:59:16.413634 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2882a8-2784-46f4-99fe-cd8a75cfb6e5 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:16.424505 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2882a8-2784-46f4-99fe-cd8a75cfb6e5 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:16.429153 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab58cd0-0d18-4606-b56e-d61026b1b200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:16.429759 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab58cd0-0d18-4606-b56e-d61026b1b200 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:16.430041 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab58cd0-0d18-4606-b56e-d61026b1b200 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:16.430878 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab58cd0-0d18-4606-b56e-d61026b1b200 admin admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:16.431576 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab58cd0-0d18-4606-b56e-d61026b1b200 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:59:16.431910 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab58cd0-0d18-4606-b56e-d61026b1b200 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:16.433523 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab58cd0-0d18-4606-b56e-d61026b1b200 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:16.445356 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1290/2605] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:59:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:59:16.449999 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8de00a-c9e6-4cef-ad06-309db6685106 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:16.460777 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8de00a-c9e6-4cef-ad06-309db6685106 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:16.465500 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19520bce-80ef-4e94-b667-97fcc708473e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:16.466104 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19520bce-80ef-4e94-b667-97fcc708473e admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:16.466417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19520bce-80ef-4e94-b667-97fcc708473e admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:16.466657 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19520bce-80ef-4e94-b667-97fcc708473e admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:16.467310 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19520bce-80ef-4e94-b667-97fcc708473e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:16.469328 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19520bce-80ef-4e94-b667-97fcc708473e admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:16.499130 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1316/2606] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:59:16 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:59:16.503247 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab58cd0-0d18-4606-b56e-d61026b1b200 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:16.514320 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab58cd0-0d18-4606-b56e-d61026b1b200 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:16.572392 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f881176d-cdc5-43ca-9e6e-7a4b64c7c12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:16.573151 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881176d-cdc5-43ca-9e6e-7a4b64c7c12a admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:16.573560 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881176d-cdc5-43ca-9e6e-7a4b64c7c12a admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:16.573690 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881176d-cdc5-43ca-9e6e-7a4b64c7c12a admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:16.574264 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f881176d-cdc5-43ca-9e6e-7a4b64c7c12a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:16.576236 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f881176d-cdc5-43ca-9e6e-7a4b64c7c12a admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:16.604232 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1291/2607] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:59:16 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:59:16.605718 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19520bce-80ef-4e94-b667-97fcc708473e admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:16.616819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19520bce-80ef-4e94-b667-97fcc708473e admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:16.622540 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eab6a2-0189-4f07-b09f-6973ff23493f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:16.623031 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eab6a2-0189-4f07-b09f-6973ff23493f admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:16.623323 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eab6a2-0189-4f07-b09f-6973ff23493f admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:16.623596 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eab6a2-0189-4f07-b09f-6973ff23493f admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:16.624350 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eab6a2-0189-4f07-b09f-6973ff23493f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:16.625982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eab6a2-0189-4f07-b09f-6973ff23493f admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:16.637622 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1317/2608] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:59:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:59:16.641775 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f881176d-cdc5-43ca-9e6e-7a4b64c7c12a admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:16.653120 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f881176d-cdc5-43ca-9e6e-7a4b64c7c12a admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:16.659628 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b072c9-d6bb-4e51-bb78-1a9265ddba1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:16.660443 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b072c9-d6bb-4e51-bb78-1a9265ddba1d admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:16.660730 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b072c9-d6bb-4e51-bb78-1a9265ddba1d admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:16.661021 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b072c9-d6bb-4e51-bb78-1a9265ddba1d admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:16.661764 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b072c9-d6bb-4e51-bb78-1a9265ddba1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:16.663387 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b072c9-d6bb-4e51-bb78-1a9265ddba1d admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:16.665289 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:59:16.681963 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1292/2609] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:59:16 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:59:16.685693 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eab6a2-0189-4f07-b09f-6973ff23493f admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:16.698344 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eab6a2-0189-4f07-b09f-6973ff23493f admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:16.704804 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305ed3c-834f-4c33-b1c1-f4db470361b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:16.705478 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305ed3c-834f-4c33-b1c1-f4db470361b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:16.705760 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305ed3c-834f-4c33-b1c1-f4db470361b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:16.706039 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305ed3c-834f-4c33-b1c1-f4db470361b4 admin admin] PATH_INFO: `/v3/projects/8723abda23a048dd930f40bdb6e75ac4/users/5a5ed0cfa9da4f9a94ea7ce793f09bbb/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:16.723244 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1305ed3c-834f-4c33-b1c1-f4db470361b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8723abda23a048dd930f40bdb6e75ac4, user_id=5a5ed0cfa9da4f9a94ea7ce793f09bbb, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:16.725956 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1305ed3c-834f-4c33-b1c1-f4db470361b4 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:16.739350 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1318/2610] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:59:16 2022] PUT /identity/v3/projects/8723abda23a048dd930f40bdb6e75ac4/users/5a5ed0cfa9da4f9a94ea7ce793f09bbb/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:59:16.750158 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16940172-ea1f-47de-8794-24aa99dab3b8 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:16.750470 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16940172-ea1f-47de-8794-24aa99dab3b8 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:16.750771 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16940172-ea1f-47de-8794-24aa99dab3b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:16.789149 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-16940172-ea1f-47de-8794-24aa99dab3b8 None None] MFA Rules not processed for user `5a5ed0cfa9da4f9a94ea7ce793f09bbb`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:59:16.798392 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-16940172-ea1f-47de-8794-24aa99dab3b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:16.878876 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1293/2611] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:59:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:59:16.883982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305ed3c-834f-4c33-b1c1-f4db470361b4 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:16.932987 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:59:16.962896 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305ed3c-834f-4c33-b1c1-f4db470361b4 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:16.967884 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c46650c-3e1e-4bda-80d8-07f9ceea47c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:16.968679 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46650c-3e1e-4bda-80d8-07f9ceea47c2 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:16.969057 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46650c-3e1e-4bda-80d8-07f9ceea47c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:16.969433 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46650c-3e1e-4bda-80d8-07f9ceea47c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:16.970209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c46650c-3e1e-4bda-80d8-07f9ceea47c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:16.972330 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c46650c-3e1e-4bda-80d8-07f9ceea47c2 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:17.004550 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1319/2612] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:59:16 2022] POST /identity/v3/projects => generated 377 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:59:17.008971 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16940172-ea1f-47de-8794-24aa99dab3b8 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:17.021919 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16940172-ea1f-47de-8794-24aa99dab3b8 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:17.089405 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e670ee0-960f-4a89-834e-15d97e31ba43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:17.090022 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e670ee0-960f-4a89-834e-15d97e31ba43 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:17.090296 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e670ee0-960f-4a89-834e-15d97e31ba43 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:17.090566 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e670ee0-960f-4a89-834e-15d97e31ba43 admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:17.091268 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e670ee0-960f-4a89-834e-15d97e31ba43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:17.093293 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e670ee0-960f-4a89-834e-15d97e31ba43 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:17.120752 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1294/2613] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:59:17 2022] POST /identity/v3/users => generated 289 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 12:59:17.124885 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c46650c-3e1e-4bda-80d8-07f9ceea47c2 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:17.136513 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c46650c-3e1e-4bda-80d8-07f9ceea47c2 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:17.141175 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002d3c6e-0a51-40fd-86ba-700232e086bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:17.141806 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d3c6e-0a51-40fd-86ba-700232e086bf admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:17.142084 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d3c6e-0a51-40fd-86ba-700232e086bf admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:17.142356 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d3c6e-0a51-40fd-86ba-700232e086bf admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:17.143059 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002d3c6e-0a51-40fd-86ba-700232e086bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:17.144714 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002d3c6e-0a51-40fd-86ba-700232e086bf admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:17.156106 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1320/2614] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:59:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:59:17.159985 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e670ee0-960f-4a89-834e-15d97e31ba43 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:17.175976 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e670ee0-960f-4a89-834e-15d97e31ba43 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:17.181784 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b12fcc3-e081-48a2-babb-f734700a873e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:17.182386 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b12fcc3-e081-48a2-babb-f734700a873e admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:17.182672 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b12fcc3-e081-48a2-babb-f734700a873e admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:17.182946 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b12fcc3-e081-48a2-babb-f734700a873e admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:17.183678 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b12fcc3-e081-48a2-babb-f734700a873e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:17.185326 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b12fcc3-e081-48a2-babb-f734700a873e admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:17.198489 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1295/2615] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:59:17 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:59:17.202151 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002d3c6e-0a51-40fd-86ba-700232e086bf admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:17.213072 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002d3c6e-0a51-40fd-86ba-700232e086bf admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:17.219820 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406bef01-7cee-41fa-b764-af451c1951b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:17.220505 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406bef01-7cee-41fa-b764-af451c1951b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:17.220784 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406bef01-7cee-41fa-b764-af451c1951b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:17.221056 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406bef01-7cee-41fa-b764-af451c1951b5 admin admin] PATH_INFO: `/v3/projects/402c7dbf641b4e4f86881fda04ee2cd1/users/94f83053457a42f4a049b5021e6adcd3/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:17.264107 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406bef01-7cee-41fa-b764-af451c1951b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=402c7dbf641b4e4f86881fda04ee2cd1, user_id=94f83053457a42f4a049b5021e6adcd3, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:17.269039 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406bef01-7cee-41fa-b764-af451c1951b5 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:17.301500 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1321/2616] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:59:17 2022] PUT /identity/v3/projects/402c7dbf641b4e4f86881fda04ee2cd1/users/94f83053457a42f4a049b5021e6adcd3/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:59:17.313761 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a66b64-b877-4d6b-91e1-771cb9330152 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:17.314186 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a66b64-b877-4d6b-91e1-771cb9330152 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:17.314588 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a66b64-b877-4d6b-91e1-771cb9330152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:17.350779 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-f3a66b64-b877-4d6b-91e1-771cb9330152 None None] MFA Rules not processed for user `94f83053457a42f4a049b5021e6adcd3`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:59:17.353507 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-f3a66b64-b877-4d6b-91e1-771cb9330152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:17.449838 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:59:17.462589 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1296/2617] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:59:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:59:17.466999 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406bef01-7cee-41fa-b764-af451c1951b5 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:17.547980 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406bef01-7cee-41fa-b764-af451c1951b5 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:17.552942 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e983c881-384c-47f4-8225-ff3ae399f8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:17.553568 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e983c881-384c-47f4-8225-ff3ae399f8b6 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:17.553848 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e983c881-384c-47f4-8225-ff3ae399f8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:17.554120 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e983c881-384c-47f4-8225-ff3ae399f8b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:17.554794 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e983c881-384c-47f4-8225-ff3ae399f8b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:17.556756 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e983c881-384c-47f4-8225-ff3ae399f8b6 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:17.585041 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1322/2618] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:59:17 2022] POST /identity/v3/projects => generated 377 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:59:17.589618 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a66b64-b877-4d6b-91e1-771cb9330152 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:17.602857 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a66b64-b877-4d6b-91e1-771cb9330152 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:17.661785 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a71df-dc7b-4eb9-9d25-bd61ea724040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:17.662307 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a71df-dc7b-4eb9-9d25-bd61ea724040 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:17.662584 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a71df-dc7b-4eb9-9d25-bd61ea724040 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:17.662856 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a71df-dc7b-4eb9-9d25-bd61ea724040 admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:17.663569 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997a71df-dc7b-4eb9-9d25-bd61ea724040 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:17.666148 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997a71df-dc7b-4eb9-9d25-bd61ea724040 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:17.686983 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1297/2619] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:59:17 2022] POST /identity/v3/users => generated 295 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:59:17.691020 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e983c881-384c-47f4-8225-ff3ae399f8b6 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:17.702067 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e983c881-384c-47f4-8225-ff3ae399f8b6 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:17.706732 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03888bb5-37ac-4263-96d8-93c9aac608d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:17.707340 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03888bb5-37ac-4263-96d8-93c9aac608d6 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:17.707623 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03888bb5-37ac-4263-96d8-93c9aac608d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:17.707922 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03888bb5-37ac-4263-96d8-93c9aac608d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:17.708612 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03888bb5-37ac-4263-96d8-93c9aac608d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:17.710219 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03888bb5-37ac-4263-96d8-93c9aac608d6 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:17.721517 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1323/2620] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:59:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:59:17.726689 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a71df-dc7b-4eb9-9d25-bd61ea724040 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:17.737466 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a71df-dc7b-4eb9-9d25-bd61ea724040 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:17.742190 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7060feb0-d080-4edb-a141-67b87949b007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:17.742876 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060feb0-d080-4edb-a141-67b87949b007 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:17.743161 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060feb0-d080-4edb-a141-67b87949b007 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:17.743442 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060feb0-d080-4edb-a141-67b87949b007 admin admin] PATH_INFO: `/v3/projects/57ca87e9c39a4856849fce307341a2c2/users/d2d5ae9724ab4fd6814cc3f681f6c5af/roles/0e78444e43c744668178971af4087ca0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:17.758922 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7060feb0-d080-4edb-a141-67b87949b007 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57ca87e9c39a4856849fce307341a2c2, user_id=d2d5ae9724ab4fd6814cc3f681f6c5af, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:17.761661 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7060feb0-d080-4edb-a141-67b87949b007 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:17.775580 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1298/2621] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:59:17 2022] PUT /identity/v3/projects/57ca87e9c39a4856849fce307341a2c2/users/d2d5ae9724ab4fd6814cc3f681f6c5af/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:59:17.779619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03888bb5-37ac-4263-96d8-93c9aac608d6 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:17.789471 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03888bb5-37ac-4263-96d8-93c9aac608d6 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:17.848629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12bfabf-5da2-489c-93bd-9ab4246e0ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:17.849251 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12bfabf-5da2-489c-93bd-9ab4246e0ef1 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:17.849560 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12bfabf-5da2-489c-93bd-9ab4246e0ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:17.849832 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12bfabf-5da2-489c-93bd-9ab4246e0ef1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:17.850526 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12bfabf-5da2-489c-93bd-9ab4246e0ef1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:17.852847 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12bfabf-5da2-489c-93bd-9ab4246e0ef1 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:17.854725 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:59:17.870299 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7060feb0-d080-4edb-a141-67b87949b007 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:17.871818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1324/2622] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:59:17 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:59:17.886610 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7060feb0-d080-4edb-a141-67b87949b007 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:17.894654 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f2ea3e-cc31-4c81-aa21-35df9e5985dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:17.895408 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f2ea3e-cc31-4c81-aa21-35df9e5985dd admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:17.895789 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f2ea3e-cc31-4c81-aa21-35df9e5985dd admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:17.895885 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f2ea3e-cc31-4c81-aa21-35df9e5985dd admin admin] PATH_INFO: `/v3/projects/57ca87e9c39a4856849fce307341a2c2/users/d2d5ae9724ab4fd6814cc3f681f6c5af/roles/0e78444e43c744668178971af4087ca0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:17.899574 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f2ea3e-cc31-4c81-aa21-35df9e5985dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=57ca87e9c39a4856849fce307341a2c2, user_id=d2d5ae9724ab4fd6814cc3f681f6c5af, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:17.902487 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f2ea3e-cc31-4c81-aa21-35df9e5985dd admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:17.921752 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1299/2623] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:59:17 2022] PUT /identity/v3/projects/57ca87e9c39a4856849fce307341a2c2/users/d2d5ae9724ab4fd6814cc3f681f6c5af/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:59:17.933327 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d5acf-74a4-4379-a421-0287bbfb80fe None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:17.933612 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d5acf-74a4-4379-a421-0287bbfb80fe None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:17.933888 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d5acf-74a4-4379-a421-0287bbfb80fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:17.964023 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-d59d5acf-74a4-4379-a421-0287bbfb80fe None None] MFA Rules not processed for user `d2d5ae9724ab4fd6814cc3f681f6c5af`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:59:17.965230 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-d59d5acf-74a4-4379-a421-0287bbfb80fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:18.042011 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1325/2624] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:59:17 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:59:18.053611 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f2ea3e-cc31-4c81-aa21-35df9e5985dd admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:18.055988 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-99f2ea3e-cc31-4c81-aa21-35df9e5985dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:18.113379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f2ea3e-cc31-4c81-aa21-35df9e5985dd admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:18.117143 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5431412-38c7-460e-b86f-35c7808206ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:18.117867 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5431412-38c7-460e-b86f-35c7808206ce service cinder] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:18.118135 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5431412-38c7-460e-b86f-35c7808206ce service cinder] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:18.118429 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5431412-38c7-460e-b86f-35c7808206ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:18.127620 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5431412-38c7-460e-b86f-35c7808206ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:18.129089 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5431412-38c7-460e-b86f-35c7808206ce service cinder] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:18.138865 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1300/2625] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:59:18 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:59:18.182478 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d5acf-74a4-4379-a421-0287bbfb80fe None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:18.183969 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-d59d5acf-74a4-4379-a421-0287bbfb80fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:18.196238 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d5acf-74a4-4379-a421-0287bbfb80fe None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:18.200035 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d9cb83-0a83-4e65-b48b-bf26672ecb67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:18.200651 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9cb83-0a83-4e65-b48b-bf26672ecb67 service cinder] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:18.200931 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9cb83-0a83-4e65-b48b-bf26672ecb67 service cinder] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:18.201204 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9cb83-0a83-4e65-b48b-bf26672ecb67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:18.259097 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d9cb83-0a83-4e65-b48b-bf26672ecb67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:18.260555 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d9cb83-0a83-4e65-b48b-bf26672ecb67 service cinder] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:18.269447 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1326/2626] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:59:18 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:59:23.301009 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5431412-38c7-460e-b86f-35c7808206ce service cinder] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:23.356655 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:59:23.372000 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5431412-38c7-460e-b86f-35c7808206ce service cinder] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:23.376564 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7103f94c-48fd-4532-b2ff-dc937d238a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:23.377242 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7103f94c-48fd-4532-b2ff-dc937d238a03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:23.377521 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7103f94c-48fd-4532-b2ff-dc937d238a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:23.378222 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7103f94c-48fd-4532-b2ff-dc937d238a03 admin admin] PATH_INFO: `/v3/users/5a5ed0cfa9da4f9a94ea7ce793f09bbb` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:23.378943 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7103f94c-48fd-4532-b2ff-dc937d238a03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a5ed0cfa9da4f9a94ea7ce793f09bbb)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:23.381491 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7103f94c-48fd-4532-b2ff-dc937d238a03 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:23.426701 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-7103f94c-48fd-4532-b2ff-dc937d238a03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a5ed0cfa9da4f9a94ea7ce793f09bbb'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:23.427367 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-7103f94c-48fd-4532-b2ff-dc937d238a03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a5ed0cfa9da4f9a94ea7ce793f09bbb'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:23.427994 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-7103f94c-48fd-4532-b2ff-dc937d238a03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a5ed0cfa9da4f9a94ea7ce793f09bbb'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:23.440339 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-7103f94c-48fd-4532-b2ff-dc937d238a03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a5ed0cfa9da4f9a94ea7ce793f09bbb'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:23.447273 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-7103f94c-48fd-4532-b2ff-dc937d238a03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a5ed0cfa9da4f9a94ea7ce793f09bbb'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:23.459993 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1301/2627] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:59:23 2022] DELETE /identity/v3/users/5a5ed0cfa9da4f9a94ea7ce793f09bbb => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:59:23.464563 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0172f23-452a-499e-9300-85af4381905e None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:23.465077 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0172f23-452a-499e-9300-85af4381905e None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:23.465077 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0172f23-452a-499e-9300-85af4381905e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:23.483760 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-b0172f23-452a-499e-9300-85af4381905e None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:59:23.484942 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-b0172f23-452a-499e-9300-85af4381905e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:23.553346 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b0bac0-a9d5-4c0c-b7a2-16c035e19369 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:23.553635 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b0bac0-a9d5-4c0c-b7a2-16c035e19369 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:23.553923 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b0bac0-a9d5-4c0c-b7a2-16c035e19369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:23.567713 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1327/2628] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:59:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:59:23.577508 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0172f23-452a-499e-9300-85af4381905e None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:23.578984 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-b0172f23-452a-499e-9300-85af4381905e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:23.584736 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-e0b0bac0-a9d5-4c0c-b7a2-16c035e19369 None None] MFA Rules not processed for user `30fc532f464843a3ad35e4d3c387bb3a`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:59:23.585951 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-e0b0bac0-a9d5-4c0c-b7a2-16c035e19369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:23.603358 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:59:23.644934 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0172f23-452a-499e-9300-85af4381905e None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:23.649255 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a94669-7165-452b-80d9-0a587ef8a3ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:23.649869 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a94669-7165-452b-80d9-0a587ef8a3ae service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:23.650145 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a94669-7165-452b-80d9-0a587ef8a3ae service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:23.650414 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a94669-7165-452b-80d9-0a587ef8a3ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:23.659945 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a94669-7165-452b-80d9-0a587ef8a3ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:23.661367 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a94669-7165-452b-80d9-0a587ef8a3ae service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:23.667775 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1302/2629] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Jul 26 12:59:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 12:59:23.672287 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1328/2630] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:59:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 12:59:23.717414 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b0bac0-a9d5-4c0c-b7a2-16c035e19369 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:23.720538 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-e0b0bac0-a9d5-4c0c-b7a2-16c035e19369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:23.780612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:59:23.797250 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b0bac0-a9d5-4c0c-b7a2-16c035e19369 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:23.802527 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2125ef7f-5901-48d2-ab2c-75b307ca8675 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd2483f74d7d48b1a7b8ffe32f63abde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:23.803146 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125ef7f-5901-48d2-ab2c-75b307ca8675 service swift] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:23.803425 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125ef7f-5901-48d2-ab2c-75b307ca8675 service swift] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:23.803737 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125ef7f-5901-48d2-ab2c-75b307ca8675 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:23.815704 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2125ef7f-5901-48d2-ab2c-75b307ca8675 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:23.817627 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2125ef7f-5901-48d2-ab2c-75b307ca8675 service swift] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:23.830551 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1303/2631] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Jul 26 12:59:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 12:59:24.005182 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a94669-7165-452b-80d9-0a587ef8a3ae service neutron] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:24.007118 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-52a94669-7165-452b-80d9-0a587ef8a3ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:24.041500 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a94669-7165-452b-80d9-0a587ef8a3ae service neutron] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:24.048650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac854e9b-d4dc-4159-a6ff-dfd0d8f756f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:24.049434 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac854e9b-d4dc-4159-a6ff-dfd0d8f756f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:24.049872 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac854e9b-d4dc-4159-a6ff-dfd0d8f756f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:24.050196 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac854e9b-d4dc-4159-a6ff-dfd0d8f756f2 admin admin] PATH_INFO: `/v3/projects/8723abda23a048dd930f40bdb6e75ac4` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:24.051308 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac854e9b-d4dc-4159-a6ff-dfd0d8f756f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8723abda23a048dd930f40bdb6e75ac4)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:24.053295 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac854e9b-d4dc-4159-a6ff-dfd0d8f756f2 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:24.110231 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-ac854e9b-d4dc-4159-a6ff-dfd0d8f756f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8723abda23a048dd930f40bdb6e75ac4', 'request_id': 'req-ac854e9b-d4dc-4159-a6ff-dfd0d8f756f2'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:24.118134 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-ac854e9b-d4dc-4159-a6ff-dfd0d8f756f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8723abda23a048dd930f40bdb6e75ac4', 'request_id': 'req-ac854e9b-d4dc-4159-a6ff-dfd0d8f756f2'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:24.127429 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-ac854e9b-d4dc-4159-a6ff-dfd0d8f756f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8723abda23a048dd930f40bdb6e75ac4', 'request_id': 'req-ac854e9b-d4dc-4159-a6ff-dfd0d8f756f2'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:24.141697 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-ac854e9b-d4dc-4159-a6ff-dfd0d8f756f2 admin admin] The token cache is being invalidate because project 8723abda23a048dd930f40bdb6e75ac4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:59:24.142304 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-ac854e9b-d4dc-4159-a6ff-dfd0d8f756f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:24.143284 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-ac854e9b-d4dc-4159-a6ff-dfd0d8f756f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:24.145287 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1329/2632] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:59:24 2022] DELETE /identity/v3/projects/8723abda23a048dd930f40bdb6e75ac4 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:59:24.151310 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2125ef7f-5901-48d2-ab2c-75b307ca8675 service swift] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:24.165060 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2125ef7f-5901-48d2-ab2c-75b307ca8675 service swift] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:24.268377 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b9e158-0170-4cb1-9742-43a04b09bb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:24.269529 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9e158-0170-4cb1-9742-43a04b09bb81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:24.270129 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9e158-0170-4cb1-9742-43a04b09bb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:24.270728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9e158-0170-4cb1-9742-43a04b09bb81 admin admin] PATH_INFO: `/v3/users/94f83053457a42f4a049b5021e6adcd3` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:24.272987 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b9e158-0170-4cb1-9742-43a04b09bb81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94f83053457a42f4a049b5021e6adcd3)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:24.275601 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b9e158-0170-4cb1-9742-43a04b09bb81 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:24.337468 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-40b9e158-0170-4cb1-9742-43a04b09bb81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94f83053457a42f4a049b5021e6adcd3'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:24.338552 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-40b9e158-0170-4cb1-9742-43a04b09bb81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94f83053457a42f4a049b5021e6adcd3'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:24.339298 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-40b9e158-0170-4cb1-9742-43a04b09bb81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94f83053457a42f4a049b5021e6adcd3'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:24.354650 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-40b9e158-0170-4cb1-9742-43a04b09bb81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94f83053457a42f4a049b5021e6adcd3'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:24.368596 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-40b9e158-0170-4cb1-9742-43a04b09bb81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94f83053457a42f4a049b5021e6adcd3'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:24.391751 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1304/2633] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:59:24 2022] DELETE /identity/v3/users/94f83053457a42f4a049b5021e6adcd3 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:59:25.166555 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac854e9b-d4dc-4159-a6ff-dfd0d8f756f2 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:25.169145 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-ac854e9b-d4dc-4159-a6ff-dfd0d8f756f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:25.202470 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac854e9b-d4dc-4159-a6ff-dfd0d8f756f2 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:25.290830 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af172661-8550-4ad5-aae1-8fbed0631287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:25.291409 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af172661-8550-4ad5-aae1-8fbed0631287 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:25.291715 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af172661-8550-4ad5-aae1-8fbed0631287 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:25.292016 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af172661-8550-4ad5-aae1-8fbed0631287 admin admin] PATH_INFO: `/v3/projects/402c7dbf641b4e4f86881fda04ee2cd1` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:25.293384 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af172661-8550-4ad5-aae1-8fbed0631287 admin admin] RBAC: Authorizing `identity:delete_project(project_id=402c7dbf641b4e4f86881fda04ee2cd1)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:25.295552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af172661-8550-4ad5-aae1-8fbed0631287 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:25.359618 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:59:25.362990 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-af172661-8550-4ad5-aae1-8fbed0631287 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '402c7dbf641b4e4f86881fda04ee2cd1', 'request_id': 'req-af172661-8550-4ad5-aae1-8fbed0631287'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:25.375499 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-af172661-8550-4ad5-aae1-8fbed0631287 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '402c7dbf641b4e4f86881fda04ee2cd1', 'request_id': 'req-af172661-8550-4ad5-aae1-8fbed0631287'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:25.384494 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-af172661-8550-4ad5-aae1-8fbed0631287 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '402c7dbf641b4e4f86881fda04ee2cd1', 'request_id': 'req-af172661-8550-4ad5-aae1-8fbed0631287'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:25.404911 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-af172661-8550-4ad5-aae1-8fbed0631287 admin admin] The token cache is being invalidate because project 402c7dbf641b4e4f86881fda04ee2cd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:59:25.405440 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-af172661-8550-4ad5-aae1-8fbed0631287 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:25.406349 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-af172661-8550-4ad5-aae1-8fbed0631287 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:25.408657 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1330/2634] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:59:25 2022] DELETE /identity/v3/projects/402c7dbf641b4e4f86881fda04ee2cd1 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:59:25.413011 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b9e158-0170-4cb1-9742-43a04b09bb81 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:25.415082 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-40b9e158-0170-4cb1-9742-43a04b09bb81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:25.434138 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b9e158-0170-4cb1-9742-43a04b09bb81 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:25.523576 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02278683-d715-4d80-b4b1-ac531766dfec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:25.524859 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02278683-d715-4d80-b4b1-ac531766dfec admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:25.525161 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02278683-d715-4d80-b4b1-ac531766dfec admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:25.525429 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02278683-d715-4d80-b4b1-ac531766dfec admin admin] PATH_INFO: `/v3/users/d2d5ae9724ab4fd6814cc3f681f6c5af` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:25.526780 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02278683-d715-4d80-b4b1-ac531766dfec admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2d5ae9724ab4fd6814cc3f681f6c5af)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:25.528713 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02278683-d715-4d80-b4b1-ac531766dfec admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:25.537005 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:59:25.594379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-02278683-d715-4d80-b4b1-ac531766dfec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2d5ae9724ab4fd6814cc3f681f6c5af'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:25.595467 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-02278683-d715-4d80-b4b1-ac531766dfec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2d5ae9724ab4fd6814cc3f681f6c5af'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:25.597203 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-02278683-d715-4d80-b4b1-ac531766dfec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2d5ae9724ab4fd6814cc3f681f6c5af'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:25.615376 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-02278683-d715-4d80-b4b1-ac531766dfec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2d5ae9724ab4fd6814cc3f681f6c5af'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:25.623325 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-02278683-d715-4d80-b4b1-ac531766dfec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2d5ae9724ab4fd6814cc3f681f6c5af'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:25.640664 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1305/2635] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:59:25 2022] DELETE /identity/v3/users/d2d5ae9724ab4fd6814cc3f681f6c5af => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:59:25.995799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af172661-8550-4ad5-aae1-8fbed0631287 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:25.997893 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-af172661-8550-4ad5-aae1-8fbed0631287 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:26.013614 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af172661-8550-4ad5-aae1-8fbed0631287 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:26.109336 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a3c236-fe1c-4745-b5cd-a170995246d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:26.110271 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a3c236-fe1c-4745-b5cd-a170995246d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:26.110558 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a3c236-fe1c-4745-b5cd-a170995246d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:26.110830 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a3c236-fe1c-4745-b5cd-a170995246d3 admin admin] PATH_INFO: `/v3/projects/57ca87e9c39a4856849fce307341a2c2` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:26.112156 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a3c236-fe1c-4745-b5cd-a170995246d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57ca87e9c39a4856849fce307341a2c2)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:26.114244 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a3c236-fe1c-4745-b5cd-a170995246d3 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:26.195367 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-e3a3c236-fe1c-4745-b5cd-a170995246d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57ca87e9c39a4856849fce307341a2c2', 'request_id': 'req-e3a3c236-fe1c-4745-b5cd-a170995246d3'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:26.204030 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-e3a3c236-fe1c-4745-b5cd-a170995246d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57ca87e9c39a4856849fce307341a2c2', 'request_id': 'req-e3a3c236-fe1c-4745-b5cd-a170995246d3'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:26.215761 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-e3a3c236-fe1c-4745-b5cd-a170995246d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57ca87e9c39a4856849fce307341a2c2', 'request_id': 'req-e3a3c236-fe1c-4745-b5cd-a170995246d3'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:26.229159 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-e3a3c236-fe1c-4745-b5cd-a170995246d3 admin admin] The token cache is being invalidate because project 57ca87e9c39a4856849fce307341a2c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:59:26.229543 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-e3a3c236-fe1c-4745-b5cd-a170995246d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:26.230196 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-e3a3c236-fe1c-4745-b5cd-a170995246d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:26.231916 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1331/2636] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:59:25 2022] DELETE /identity/v3/projects/57ca87e9c39a4856849fce307341a2c2 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:59:26.238044 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8250a82-aac0-4065-962b-00a51d7561bd None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:26.238332 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8250a82-aac0-4065-962b-00a51d7561bd None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:26.238611 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8250a82-aac0-4065-962b-00a51d7561bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:26.265599 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-b8250a82-aac0-4065-962b-00a51d7561bd None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:59:26.267030 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-b8250a82-aac0-4065-962b-00a51d7561bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:26.357158 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1306/2637] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:59:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:59:26.365648 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72718d-aeb4-4a73-9573-d4ae242126fe None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:26.365940 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72718d-aeb4-4a73-9573-d4ae242126fe None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:26.366246 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72718d-aeb4-4a73-9573-d4ae242126fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:26.405120 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-bf72718d-aeb4-4a73-9573-d4ae242126fe None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:59:26.406355 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-bf72718d-aeb4-4a73-9573-d4ae242126fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:26.419573 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1332/2638] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:59:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:59:26.420818 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8250a82-aac0-4065-962b-00a51d7561bd None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:26.432701 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8250a82-aac0-4065-962b-00a51d7561bd None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:26.438329 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77755340-b08a-49e0-a942-02b8027eefd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:26.439031 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77755340-b08a-49e0-a942-02b8027eefd9 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:26.439379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77755340-b08a-49e0-a942-02b8027eefd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:26.439753 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77755340-b08a-49e0-a942-02b8027eefd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:26.440946 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77755340-b08a-49e0-a942-02b8027eefd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:59:26.440946 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77755340-b08a-49e0-a942-02b8027eefd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:26.445226 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77755340-b08a-49e0-a942-02b8027eefd9 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:26.463056 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1307/2639] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:59:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 12:59:26.467479 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf72718d-aeb4-4a73-9573-d4ae242126fe None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:26.480652 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf72718d-aeb4-4a73-9573-d4ae242126fe None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:26.485624 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79a6199-3435-415e-b75c-bf0f33992835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:26.486252 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a6199-3435-415e-b75c-bf0f33992835 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:26.486533 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a6199-3435-415e-b75c-bf0f33992835 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:26.486809 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a6199-3435-415e-b75c-bf0f33992835 admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:26.487485 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79a6199-3435-415e-b75c-bf0f33992835 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:26.489441 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79a6199-3435-415e-b75c-bf0f33992835 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:26.540265 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1333/2640] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:59:26 2022] POST /identity/v3/projects => generated 373 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:59:26.550013 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77755340-b08a-49e0-a942-02b8027eefd9 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:26.564091 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77755340-b08a-49e0-a942-02b8027eefd9 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:26.635477 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f87f37-7ce1-4873-b96f-53647e7651a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:26.636153 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f87f37-7ce1-4873-b96f-53647e7651a0 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:26.636434 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f87f37-7ce1-4873-b96f-53647e7651a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:26.636719 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f87f37-7ce1-4873-b96f-53647e7651a0 admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:26.637747 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f87f37-7ce1-4873-b96f-53647e7651a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:26.640442 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f87f37-7ce1-4873-b96f-53647e7651a0 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:26.671761 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1308/2641] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:59:26 2022] POST /identity/v3/users => generated 287 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:59:26.675415 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79a6199-3435-415e-b75c-bf0f33992835 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:26.688345 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79a6199-3435-415e-b75c-bf0f33992835 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:26.699680 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8495b16c-c8ea-498a-8e99-007e93ac663d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:26.699680 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495b16c-c8ea-498a-8e99-007e93ac663d admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:26.699680 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495b16c-c8ea-498a-8e99-007e93ac663d admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:26.699680 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495b16c-c8ea-498a-8e99-007e93ac663d admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:26.700342 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8495b16c-c8ea-498a-8e99-007e93ac663d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:26.701558 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8495b16c-c8ea-498a-8e99-007e93ac663d admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:26.714975 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1334/2642] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:59:26.719303 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f87f37-7ce1-4873-b96f-53647e7651a0 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:26.724822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:59:26.733317 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f87f37-7ce1-4873-b96f-53647e7651a0 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:26.746443 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6689c18-de51-4477-b981-f3ae829e524c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:26.747305 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6689c18-de51-4477-b981-f3ae829e524c admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:26.747617 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6689c18-de51-4477-b981-f3ae829e524c admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:26.747940 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6689c18-de51-4477-b981-f3ae829e524c admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:26.748737 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6689c18-de51-4477-b981-f3ae829e524c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:26.750550 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6689c18-de51-4477-b981-f3ae829e524c admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:26.775601 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1309/2643] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:59:26.776686 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8495b16c-c8ea-498a-8e99-007e93ac663d admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:26.788549 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8495b16c-c8ea-498a-8e99-007e93ac663d admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:26.798898 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4edda-2186-48d0-9121-3ad82d261cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:26.799534 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4edda-2186-48d0-9121-3ad82d261cad admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:26.799848 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4edda-2186-48d0-9121-3ad82d261cad admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:26.802523 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4edda-2186-48d0-9121-3ad82d261cad admin admin] PATH_INFO: `/v3/projects/628d11c8f10f44e696ef7ca6106f99fe/users/b30d3820355c4c8db45aac214892e3c2/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:26.827965 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac4edda-2186-48d0-9121-3ad82d261cad admin admin] RBAC: Authorizing `identity:create_grant(project_id=628d11c8f10f44e696ef7ca6106f99fe, user_id=b30d3820355c4c8db45aac214892e3c2, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:26.830689 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac4edda-2186-48d0-9121-3ad82d261cad admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:26.851743 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1335/2644] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:59:26 2022] PUT /identity/v3/projects/628d11c8f10f44e696ef7ca6106f99fe/users/b30d3820355c4c8db45aac214892e3c2/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 12:59:26.860628 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8db8c-9902-400d-b6e2-4b479d1a7435 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:26.860963 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8db8c-9902-400d-b6e2-4b479d1a7435 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:26.861268 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8db8c-9902-400d-b6e2-4b479d1a7435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:26.902656 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-a7d8db8c-9902-400d-b6e2-4b479d1a7435 None None] MFA Rules not processed for user `b30d3820355c4c8db45aac214892e3c2`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:59:26.904315 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-a7d8db8c-9902-400d-b6e2-4b479d1a7435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:27.011786 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1310/2645] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:59:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:59:27.022653 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4edda-2186-48d0-9121-3ad82d261cad admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:27.027059 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-7ac4edda-2186-48d0-9121-3ad82d261cad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:27.031913 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:59:27.111475 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4edda-2186-48d0-9121-3ad82d261cad admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:27.117031 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32909a17-461c-40cd-83f8-b5326159dd5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:27.117234 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32909a17-461c-40cd-83f8-b5326159dd5c service cinder] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:27.117789 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32909a17-461c-40cd-83f8-b5326159dd5c service cinder] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:27.118070 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32909a17-461c-40cd-83f8-b5326159dd5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:27.129423 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32909a17-461c-40cd-83f8-b5326159dd5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:27.130884 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32909a17-461c-40cd-83f8-b5326159dd5c service cinder] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:27.159797 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1336/2646] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:59:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:59:27.198047 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8db8c-9902-400d-b6e2-4b479d1a7435 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:27.297788 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8db8c-9902-400d-b6e2-4b479d1a7435 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:27.303515 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f10c072-29c2-4da6-9558-2e0317c949a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:27.304493 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f10c072-29c2-4da6-9558-2e0317c949a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:27.304859 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f10c072-29c2-4da6-9558-2e0317c949a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:27.307274 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f10c072-29c2-4da6-9558-2e0317c949a8 admin admin] PATH_INFO: `/v3/users/b30d3820355c4c8db45aac214892e3c2` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:27.309509 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f10c072-29c2-4da6-9558-2e0317c949a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b30d3820355c4c8db45aac214892e3c2)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:27.311724 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f10c072-29c2-4da6-9558-2e0317c949a8 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:27.372626 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-4f10c072-29c2-4da6-9558-2e0317c949a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b30d3820355c4c8db45aac214892e3c2'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:27.373482 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-4f10c072-29c2-4da6-9558-2e0317c949a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b30d3820355c4c8db45aac214892e3c2'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:27.374394 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-4f10c072-29c2-4da6-9558-2e0317c949a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b30d3820355c4c8db45aac214892e3c2'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:27.390705 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-4f10c072-29c2-4da6-9558-2e0317c949a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b30d3820355c4c8db45aac214892e3c2'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:27.399673 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-4f10c072-29c2-4da6-9558-2e0317c949a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b30d3820355c4c8db45aac214892e3c2'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:27.414230 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:59:27.423816 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1311/2647] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:59:27 2022] DELETE /identity/v3/users/b30d3820355c4c8db45aac214892e3c2 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:59:27.426119 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b68544-70cc-4e82-b31b-1d1c7bcb9cf8 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:27.426558 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b68544-70cc-4e82-b31b-1d1c7bcb9cf8 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:27.427179 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b68544-70cc-4e82-b31b-1d1c7bcb9cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:27.449069 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-c8b68544-70cc-4e82-b31b-1d1c7bcb9cf8 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:59:27.450302 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-c8b68544-70cc-4e82-b31b-1d1c7bcb9cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:27.520732 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1337/2648] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:59:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:59:27.534915 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f10c072-29c2-4da6-9558-2e0317c949a8 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:27.542266 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-4f10c072-29c2-4da6-9558-2e0317c949a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:27.611313 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f10c072-29c2-4da6-9558-2e0317c949a8 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:27.615379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ca6ed3-8fc6-4a31-a5f7-3a1e381ee8d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:27.616225 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca6ed3-8fc6-4a31-a5f7-3a1e381ee8d2 service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:27.616503 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca6ed3-8fc6-4a31-a5f7-3a1e381ee8d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:27.617007 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca6ed3-8fc6-4a31-a5f7-3a1e381ee8d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:27.630815 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ca6ed3-8fc6-4a31-a5f7-3a1e381ee8d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:27.632592 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ca6ed3-8fc6-4a31-a5f7-3a1e381ee8d2 service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:27.644920 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1312/2649] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:59:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:59:28.112261 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b68544-70cc-4e82-b31b-1d1c7bcb9cf8 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:28.113905 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-c8b68544-70cc-4e82-b31b-1d1c7bcb9cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:28.136277 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b68544-70cc-4e82-b31b-1d1c7bcb9cf8 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:28.142663 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27995a17-169c-4182-8c40-0ccfe0e8b7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:28.148500 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27995a17-169c-4182-8c40-0ccfe0e8b7d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:28.149391 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27995a17-169c-4182-8c40-0ccfe0e8b7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:28.149608 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27995a17-169c-4182-8c40-0ccfe0e8b7d3 admin admin] PATH_INFO: `/v3/projects/628d11c8f10f44e696ef7ca6106f99fe` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:28.151231 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27995a17-169c-4182-8c40-0ccfe0e8b7d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=628d11c8f10f44e696ef7ca6106f99fe)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:28.153228 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27995a17-169c-4182-8c40-0ccfe0e8b7d3 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:28.220853 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-27995a17-169c-4182-8c40-0ccfe0e8b7d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '628d11c8f10f44e696ef7ca6106f99fe', 'request_id': 'req-27995a17-169c-4182-8c40-0ccfe0e8b7d3'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:28.229546 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-27995a17-169c-4182-8c40-0ccfe0e8b7d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '628d11c8f10f44e696ef7ca6106f99fe', 'request_id': 'req-27995a17-169c-4182-8c40-0ccfe0e8b7d3'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:28.241533 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-27995a17-169c-4182-8c40-0ccfe0e8b7d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '628d11c8f10f44e696ef7ca6106f99fe', 'request_id': 'req-27995a17-169c-4182-8c40-0ccfe0e8b7d3'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:28.259095 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-27995a17-169c-4182-8c40-0ccfe0e8b7d3 admin admin] The token cache is being invalidate because project 628d11c8f10f44e696ef7ca6106f99fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:59:28.259362 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-27995a17-169c-4182-8c40-0ccfe0e8b7d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:28.260020 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-27995a17-169c-4182-8c40-0ccfe0e8b7d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:28.262418 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1338/2650] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:59:28 2022] DELETE /identity/v3/projects/628d11c8f10f44e696ef7ca6106f99fe => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:59:37.640326 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=26 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:59:38.255251 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:59:39.302945 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe778652-4d63-459d-8e5c-5444af632c78 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:39.303447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe778652-4d63-459d-8e5c-5444af632c78 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:39.303592 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe778652-4d63-459d-8e5c-5444af632c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:39.323170 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-fe778652-4d63-459d-8e5c-5444af632c78 None None] MFA Rules not processed for user `30fc532f464843a3ad35e4d3c387bb3a`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:59:39.324408 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-fe778652-4d63-459d-8e5c-5444af632c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:39.384258 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1313/2651] 10.222.0.20 () {58 vars in 967 bytes} [Tue Jul 26 12:59:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:59:39.397389 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27995a17-169c-4182-8c40-0ccfe0e8b7d3 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:39.398190 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-27995a17-169c-4182-8c40-0ccfe0e8b7d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:39.465958 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27995a17-169c-4182-8c40-0ccfe0e8b7d3 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:39.471098 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d0a92b-9334-4b03-9cd4-c54d1d5e4ab1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd2483f74d7d48b1a7b8ffe32f63abde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:39.471741 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0a92b-9334-4b03-9cd4-c54d1d5e4ab1 service swift] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:39.472020 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0a92b-9334-4b03-9cd4-c54d1d5e4ab1 service swift] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:39.472285 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0a92b-9334-4b03-9cd4-c54d1d5e4ab1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:39.481118 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d0a92b-9334-4b03-9cd4-c54d1d5e4ab1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:39.482671 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d0a92b-9334-4b03-9cd4-c54d1d5e4ab1 service swift] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:39.490677 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1339/2652] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Jul 26 12:59:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 12:59:41.395053 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe778652-4d63-459d-8e5c-5444af632c78 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:41.396582 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-fe778652-4d63-459d-8e5c-5444af632c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:41.462607 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe778652-4d63-459d-8e5c-5444af632c78 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:41.467270 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d93a32a-50a9-432b-9f54-128182146150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:41.467921 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93a32a-50a9-432b-9f54-128182146150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:41.468227 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93a32a-50a9-432b-9f54-128182146150 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:41.468508 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93a32a-50a9-432b-9f54-128182146150 admin admin] PATH_INFO: `/v3/users/8e8b5c62a961409baaaa101f61437f1e` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:41.469696 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d93a32a-50a9-432b-9f54-128182146150 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e8b5c62a961409baaaa101f61437f1e)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:41.471690 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d93a32a-50a9-432b-9f54-128182146150 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:41.514915 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-5d93a32a-50a9-432b-9f54-128182146150 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e8b5c62a961409baaaa101f61437f1e'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:41.515574 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-5d93a32a-50a9-432b-9f54-128182146150 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e8b5c62a961409baaaa101f61437f1e'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:41.516197 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-5d93a32a-50a9-432b-9f54-128182146150 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e8b5c62a961409baaaa101f61437f1e'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:41.528694 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-5d93a32a-50a9-432b-9f54-128182146150 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e8b5c62a961409baaaa101f61437f1e'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:41.535510 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-5d93a32a-50a9-432b-9f54-128182146150 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e8b5c62a961409baaaa101f61437f1e'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:41.550295 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1314/2653] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:59:41 2022] DELETE /identity/v3/users/8e8b5c62a961409baaaa101f61437f1e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:59:41.721255 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d0a92b-9334-4b03-9cd4-c54d1d5e4ab1 service swift] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:41.732016 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d0a92b-9334-4b03-9cd4-c54d1d5e4ab1 service swift] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:41.791201 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f13061-6a24-400d-b20a-f744df59e3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:41.791880 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f13061-6a24-400d-b20a-f744df59e3b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:41.792162 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f13061-6a24-400d-b20a-f744df59e3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:41.792436 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f13061-6a24-400d-b20a-f744df59e3b8 admin admin] PATH_INFO: `/v3/projects/dc0f5db491bd494d943de6bf1084c1a5` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:41.793571 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f13061-6a24-400d-b20a-f744df59e3b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc0f5db491bd494d943de6bf1084c1a5)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:41.795465 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f13061-6a24-400d-b20a-f744df59e3b8 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:41.840844 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-a1f13061-6a24-400d-b20a-f744df59e3b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc0f5db491bd494d943de6bf1084c1a5', 'request_id': 'req-a1f13061-6a24-400d-b20a-f744df59e3b8'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:41.846786 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-a1f13061-6a24-400d-b20a-f744df59e3b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc0f5db491bd494d943de6bf1084c1a5', 'request_id': 'req-a1f13061-6a24-400d-b20a-f744df59e3b8'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:41.854061 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-a1f13061-6a24-400d-b20a-f744df59e3b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc0f5db491bd494d943de6bf1084c1a5', 'request_id': 'req-a1f13061-6a24-400d-b20a-f744df59e3b8'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:41.864185 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-a1f13061-6a24-400d-b20a-f744df59e3b8 admin admin] The token cache is being invalidate because project dc0f5db491bd494d943de6bf1084c1a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:59:41.864484 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-a1f13061-6a24-400d-b20a-f744df59e3b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:41.865080 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-a1f13061-6a24-400d-b20a-f744df59e3b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:41.866566 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1340/2654] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:59:41 2022] DELETE /identity/v3/projects/dc0f5db491bd494d943de6bf1084c1a5 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:59:41.872897 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2969ed-46ea-419b-b3e6-bb56dd813478 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:41.873184 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2969ed-46ea-419b-b3e6-bb56dd813478 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:41.873457 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2969ed-46ea-419b-b3e6-bb56dd813478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:41.893645 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-4e2969ed-46ea-419b-b3e6-bb56dd813478 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:59:41.894846 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-4e2969ed-46ea-419b-b3e6-bb56dd813478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:41.913769 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:59:41.957483 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1315/2655] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:59:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:59:41.965544 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ffcc0-5bb6-4110-a5b8-74e1fccf4244 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:41.965801 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ffcc0-5bb6-4110-a5b8-74e1fccf4244 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:41.966218 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ffcc0-5bb6-4110-a5b8-74e1fccf4244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:41.986254 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-079ffcc0-5bb6-4110-a5b8-74e1fccf4244 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:59:41.987463 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-079ffcc0-5bb6-4110-a5b8-74e1fccf4244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:41.995911 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1341/2656] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:59:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:59:42.000224 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2969ed-46ea-419b-b3e6-bb56dd813478 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:42.010782 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2969ed-46ea-419b-b3e6-bb56dd813478 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:42.015453 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f49d5-3aff-436d-b791-5c5bacb2adeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:42.016203 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f49d5-3aff-436d-b791-5c5bacb2adeb admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:42.016379 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f49d5-3aff-436d-b791-5c5bacb2adeb admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:42.016655 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f49d5-3aff-436d-b791-5c5bacb2adeb admin admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:42.017346 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72f49d5-3aff-436d-b791-5c5bacb2adeb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:59:42.017684 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72f49d5-3aff-436d-b791-5c5bacb2adeb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:42.019260 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72f49d5-3aff-436d-b791-5c5bacb2adeb admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:42.030864 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1316/2657] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:59:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:59:42.035317 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ffcc0-5bb6-4110-a5b8-74e1fccf4244 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:42.045688 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ffcc0-5bb6-4110-a5b8-74e1fccf4244 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:42.050785 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5748a725-98cd-4d18-9788-53c6fe04d424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:42.051401 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5748a725-98cd-4d18-9788-53c6fe04d424 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:42.051715 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5748a725-98cd-4d18-9788-53c6fe04d424 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:42.051996 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5748a725-98cd-4d18-9788-53c6fe04d424 admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:42.052666 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5748a725-98cd-4d18-9788-53c6fe04d424 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:42.055109 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5748a725-98cd-4d18-9788-53c6fe04d424 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:42.068127 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:59:42.086347 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1342/2658] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:59:42 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:59:42.090621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f49d5-3aff-436d-b791-5c5bacb2adeb admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:42.100822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f49d5-3aff-436d-b791-5c5bacb2adeb admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:42.158336 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f092b4d1-c3d1-4e57-8b66-b9ff3b2ba271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:42.158951 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092b4d1-c3d1-4e57-8b66-b9ff3b2ba271 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:42.159238 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092b4d1-c3d1-4e57-8b66-b9ff3b2ba271 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:42.159512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092b4d1-c3d1-4e57-8b66-b9ff3b2ba271 admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:42.160220 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f092b4d1-c3d1-4e57-8b66-b9ff3b2ba271 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:42.162186 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f092b4d1-c3d1-4e57-8b66-b9ff3b2ba271 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:42.185650 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1317/2659] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:59:42 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:59:42.189735 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5748a725-98cd-4d18-9788-53c6fe04d424 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:42.200035 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5748a725-98cd-4d18-9788-53c6fe04d424 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:42.204725 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3745775-8fcd-47fc-84ec-7d079b720774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:42.205335 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3745775-8fcd-47fc-84ec-7d079b720774 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:42.205615 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3745775-8fcd-47fc-84ec-7d079b720774 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:42.205889 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3745775-8fcd-47fc-84ec-7d079b720774 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:42.206581 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3745775-8fcd-47fc-84ec-7d079b720774 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:42.208221 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3745775-8fcd-47fc-84ec-7d079b720774 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:42.218253 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1343/2660] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:59:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:59:42.222111 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f092b4d1-c3d1-4e57-8b66-b9ff3b2ba271 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:42.232631 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f092b4d1-c3d1-4e57-8b66-b9ff3b2ba271 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:42.237170 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a22b5-4a0d-4c54-9ff7-7148218cbea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:42.237762 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a22b5-4a0d-4c54-9ff7-7148218cbea7 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:42.238039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a22b5-4a0d-4c54-9ff7-7148218cbea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:42.238311 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a22b5-4a0d-4c54-9ff7-7148218cbea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:42.238992 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03a22b5-4a0d-4c54-9ff7-7148218cbea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:42.240644 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03a22b5-4a0d-4c54-9ff7-7148218cbea7 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:42.251748 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1318/2661] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:59:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:59:42.256168 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3745775-8fcd-47fc-84ec-7d079b720774 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:42.266702 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3745775-8fcd-47fc-84ec-7d079b720774 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:42.271436 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246d16da-092a-4329-9054-f9fdddb55009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:42.272103 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d16da-092a-4329-9054-f9fdddb55009 admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:42.272388 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d16da-092a-4329-9054-f9fdddb55009 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:42.272667 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d16da-092a-4329-9054-f9fdddb55009 admin admin] PATH_INFO: `/v3/projects/3ffe7d4eb95549d38d070fe9929fcbe1/users/04a369bfa5894a2b9858885f36fd03a2/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:42.287374 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246d16da-092a-4329-9054-f9fdddb55009 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ffe7d4eb95549d38d070fe9929fcbe1, user_id=04a369bfa5894a2b9858885f36fd03a2, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:42.290586 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246d16da-092a-4329-9054-f9fdddb55009 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:42.304272 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1344/2662] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:59:42 2022] PUT /identity/v3/projects/3ffe7d4eb95549d38d070fe9929fcbe1/users/04a369bfa5894a2b9858885f36fd03a2/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:59:42.315159 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a077365-71c2-4bf0-a134-02cec708a829 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:42.315450 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a077365-71c2-4bf0-a134-02cec708a829 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:42.315765 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a077365-71c2-4bf0-a134-02cec708a829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:42.342595 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-5a077365-71c2-4bf0-a134-02cec708a829 None None] MFA Rules not processed for user `04a369bfa5894a2b9858885f36fd03a2`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:59:42.343852 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-5a077365-71c2-4bf0-a134-02cec708a829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:42.407353 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1319/2663] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:59:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:59:42.419235 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246d16da-092a-4329-9054-f9fdddb55009 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:42.420980 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-246d16da-092a-4329-9054-f9fdddb55009 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:42.499418 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246d16da-092a-4329-9054-f9fdddb55009 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:42.503003 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba6d0ce-8e95-415c-833a-40125176f39d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:42.503608 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba6d0ce-8e95-415c-833a-40125176f39d service cinder] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:42.503913 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba6d0ce-8e95-415c-833a-40125176f39d service cinder] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:42.504181 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba6d0ce-8e95-415c-833a-40125176f39d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:42.512776 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba6d0ce-8e95-415c-833a-40125176f39d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:42.514188 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba6d0ce-8e95-415c-833a-40125176f39d service cinder] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:42.522714 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1345/2664] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:59:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:59:42.548290 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a077365-71c2-4bf0-a134-02cec708a829 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:42.569074 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:59:42.616155 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a077365-71c2-4bf0-a134-02cec708a829 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:42.620779 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5817f-0856-4958-8769-0c57062226ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:42.620779 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5817f-0856-4958-8769-0c57062226ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:42.620779 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5817f-0856-4958-8769-0c57062226ef admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:42.620779 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5817f-0856-4958-8769-0c57062226ef admin admin] PATH_INFO: `/v3/users/04a369bfa5894a2b9858885f36fd03a2` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:42.621194 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce5817f-0856-4958-8769-0c57062226ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=04a369bfa5894a2b9858885f36fd03a2)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:42.622507 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce5817f-0856-4958-8769-0c57062226ef admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:42.669706 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-4ce5817f-0856-4958-8769-0c57062226ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04a369bfa5894a2b9858885f36fd03a2'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:42.670367 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-4ce5817f-0856-4958-8769-0c57062226ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04a369bfa5894a2b9858885f36fd03a2'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:42.671025 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-4ce5817f-0856-4958-8769-0c57062226ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04a369bfa5894a2b9858885f36fd03a2'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:42.683571 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-4ce5817f-0856-4958-8769-0c57062226ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04a369bfa5894a2b9858885f36fd03a2'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:42.690594 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-4ce5817f-0856-4958-8769-0c57062226ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04a369bfa5894a2b9858885f36fd03a2'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:42.705711 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1320/2665] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 12:59:42 2022] DELETE /identity/v3/users/04a369bfa5894a2b9858885f36fd03a2 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:59:42.710312 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb10cb3-3d5f-4c20-b6a2-3ca0d8bbf9ad None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:42.710602 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb10cb3-3d5f-4c20-b6a2-3ca0d8bbf9ad None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:42.711400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb10cb3-3d5f-4c20-b6a2-3ca0d8bbf9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:42.731761 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-cdb10cb3-3d5f-4c20-b6a2-3ca0d8bbf9ad None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:59:42.732949 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-cdb10cb3-3d5f-4c20-b6a2-3ca0d8bbf9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:42.796764 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1346/2666] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:59:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:59:42.806591 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5817f-0856-4958-8769-0c57062226ef admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:42.808052 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-4ce5817f-0856-4958-8769-0c57062226ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:42.891699 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5817f-0856-4958-8769-0c57062226ef admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:42.895414 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c869c5b7-0b75-4189-a3a8-7c8493fe155c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:42.896078 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c869c5b7-0b75-4189-a3a8-7c8493fe155c service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:42.896359 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c869c5b7-0b75-4189-a3a8-7c8493fe155c service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:42.896639 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c869c5b7-0b75-4189-a3a8-7c8493fe155c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:42.908531 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c869c5b7-0b75-4189-a3a8-7c8493fe155c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:42.909990 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c869c5b7-0b75-4189-a3a8-7c8493fe155c service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:42.919788 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1321/2667] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:59:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:59:43.205352 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb10cb3-3d5f-4c20-b6a2-3ca0d8bbf9ad None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:43.207012 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-cdb10cb3-3d5f-4c20-b6a2-3ca0d8bbf9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:43.220537 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb10cb3-3d5f-4c20-b6a2-3ca0d8bbf9ad None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:43.225397 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f06777-a3ef-4caf-86c3-eb44f77cb273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:43.226194 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f06777-a3ef-4caf-86c3-eb44f77cb273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:43.226471 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f06777-a3ef-4caf-86c3-eb44f77cb273 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:43.226740 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f06777-a3ef-4caf-86c3-eb44f77cb273 admin admin] PATH_INFO: `/v3/projects/3ffe7d4eb95549d38d070fe9929fcbe1` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:43.227935 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f06777-a3ef-4caf-86c3-eb44f77cb273 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ffe7d4eb95549d38d070fe9929fcbe1)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:43.229852 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f06777-a3ef-4caf-86c3-eb44f77cb273 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:43.270062 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:59:43.281642 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-46f06777-a3ef-4caf-86c3-eb44f77cb273 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ffe7d4eb95549d38d070fe9929fcbe1', 'request_id': 'req-46f06777-a3ef-4caf-86c3-eb44f77cb273'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:43.287810 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-46f06777-a3ef-4caf-86c3-eb44f77cb273 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ffe7d4eb95549d38d070fe9929fcbe1', 'request_id': 'req-46f06777-a3ef-4caf-86c3-eb44f77cb273'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:43.295176 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-46f06777-a3ef-4caf-86c3-eb44f77cb273 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ffe7d4eb95549d38d070fe9929fcbe1', 'request_id': 'req-46f06777-a3ef-4caf-86c3-eb44f77cb273'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:43.305870 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-46f06777-a3ef-4caf-86c3-eb44f77cb273 admin admin] The token cache is being invalidate because project 3ffe7d4eb95549d38d070fe9929fcbe1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 12:59:43.306157 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-46f06777-a3ef-4caf-86c3-eb44f77cb273 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:43.306787 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-46f06777-a3ef-4caf-86c3-eb44f77cb273 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 12:59:43.308403 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1347/2668] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 12:59:43 2022] DELETE /identity/v3/projects/3ffe7d4eb95549d38d070fe9929fcbe1 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:59:43.313282 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3a8b1-1302-4913-8e58-dc7ae750a885 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:43.313563 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3a8b1-1302-4913-8e58-dc7ae750a885 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:43.313854 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3a8b1-1302-4913-8e58-dc7ae750a885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:43.333605 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-1da3a8b1-1302-4913-8e58-dc7ae750a885 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:59:43.334786 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-1da3a8b1-1302-4913-8e58-dc7ae750a885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:43.351837 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:59:43.396452 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1322/2669] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:59:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:59:43.460775 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d6e4d-75a6-4432-9c6c-d3cb9ce24d1a None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:43.461099 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d6e4d-75a6-4432-9c6c-d3cb9ce24d1a None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:43.461377 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d6e4d-75a6-4432-9c6c-d3cb9ce24d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:43.480960 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-d61d6e4d-75a6-4432-9c6c-d3cb9ce24d1a None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:59:43.482293 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-d61d6e4d-75a6-4432-9c6c-d3cb9ce24d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:43.490522 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1348/2670] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:59:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:59:43.495374 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da3a8b1-1302-4913-8e58-dc7ae750a885 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:43.505137 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da3a8b1-1302-4913-8e58-dc7ae750a885 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:43.510585 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a0c232-a6bf-4bc1-bace-66611a34671a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:43.511190 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a0c232-a6bf-4bc1-bace-66611a34671a admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:43.511497 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a0c232-a6bf-4bc1-bace-66611a34671a admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:43.511804 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a0c232-a6bf-4bc1-bace-66611a34671a admin admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:43.512497 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a0c232-a6bf-4bc1-bace-66611a34671a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 12:59:43.512805 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a0c232-a6bf-4bc1-bace-66611a34671a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:43.514403 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a0c232-a6bf-4bc1-bace-66611a34671a admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:43.525936 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1323/2671] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 12:59:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 12:59:43.530335 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61d6e4d-75a6-4432-9c6c-d3cb9ce24d1a None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:43.539550 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61d6e4d-75a6-4432-9c6c-d3cb9ce24d1a None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:43.544300 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c2a26-5046-4043-b206-e384b2129b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:43.544911 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c2a26-5046-4043-b206-e384b2129b04 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:43.545192 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c2a26-5046-4043-b206-e384b2129b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:43.545462 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c2a26-5046-4043-b206-e384b2129b04 admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:43.546350 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319c2a26-5046-4043-b206-e384b2129b04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:43.548286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319c2a26-5046-4043-b206-e384b2129b04 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:43.576900 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1349/2672] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 12:59:43 2022] POST /identity/v3/projects => generated 391 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:59:43.581057 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a0c232-a6bf-4bc1-bace-66611a34671a admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:43.591358 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a0c232-a6bf-4bc1-bace-66611a34671a admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:43.648291 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eec6c31-e16a-45b8-808e-f0a4f83140bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:43.648926 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec6c31-e16a-45b8-808e-f0a4f83140bc admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:43.649207 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec6c31-e16a-45b8-808e-f0a4f83140bc admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:43.649482 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec6c31-e16a-45b8-808e-f0a4f83140bc admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:43.650153 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eec6c31-e16a-45b8-808e-f0a4f83140bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:43.652099 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eec6c31-e16a-45b8-808e-f0a4f83140bc admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:43.673779 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1324/2673] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 12:59:43 2022] POST /identity/v3/users => generated 296 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 12:59:43.678934 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c2a26-5046-4043-b206-e384b2129b04 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:43.689793 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c2a26-5046-4043-b206-e384b2129b04 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:43.694402 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda07fc-03ac-4e85-a0fe-b58c64fda4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:43.694996 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda07fc-03ac-4e85-a0fe-b58c64fda4e9 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:43.695280 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda07fc-03ac-4e85-a0fe-b58c64fda4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:43.695557 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda07fc-03ac-4e85-a0fe-b58c64fda4e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:43.696282 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda07fc-03ac-4e85-a0fe-b58c64fda4e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:43.697885 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda07fc-03ac-4e85-a0fe-b58c64fda4e9 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:43.707735 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1350/2674] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:59:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:59:43.711586 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eec6c31-e16a-45b8-808e-f0a4f83140bc admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:43.721966 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eec6c31-e16a-45b8-808e-f0a4f83140bc admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:43.727172 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af144ebc-dc20-4da4-b3d2-5318dea5d595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:43.727808 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af144ebc-dc20-4da4-b3d2-5318dea5d595 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:43.728091 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af144ebc-dc20-4da4-b3d2-5318dea5d595 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:43.728367 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af144ebc-dc20-4da4-b3d2-5318dea5d595 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:43.729067 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af144ebc-dc20-4da4-b3d2-5318dea5d595 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:43.730691 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af144ebc-dc20-4da4-b3d2-5318dea5d595 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:43.741311 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1325/2675] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 12:59:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 12:59:43.746049 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda07fc-03ac-4e85-a0fe-b58c64fda4e9 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:43.756371 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda07fc-03ac-4e85-a0fe-b58c64fda4e9 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:43.761857 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7126ba48-42c0-48eb-9379-ca7805a974d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:43.762528 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126ba48-42c0-48eb-9379-ca7805a974d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:43.762812 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126ba48-42c0-48eb-9379-ca7805a974d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:43.763210 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126ba48-42c0-48eb-9379-ca7805a974d5 admin admin] PATH_INFO: `/v3/projects/f47ad327b2a14f0fa121208b521e47cc/users/77243319bb26419b9431bc9baf8ee871/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:43.778008 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7126ba48-42c0-48eb-9379-ca7805a974d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f47ad327b2a14f0fa121208b521e47cc, user_id=77243319bb26419b9431bc9baf8ee871, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:43.780694 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7126ba48-42c0-48eb-9379-ca7805a974d5 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:43.793641 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1351/2676] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 12:59:43 2022] PUT /identity/v3/projects/f47ad327b2a14f0fa121208b521e47cc/users/77243319bb26419b9431bc9baf8ee871/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 12:59:43.803560 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6ef4c-790d-4bec-a562-621a444fb0b6 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:43.803898 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6ef4c-790d-4bec-a562-621a444fb0b6 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:43.804178 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6ef4c-790d-4bec-a562-621a444fb0b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:43.832779 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-45e6ef4c-790d-4bec-a562-621a444fb0b6 None None] MFA Rules not processed for user `77243319bb26419b9431bc9baf8ee871`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 12:59:43.833990 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-45e6ef4c-790d-4bec-a562-621a444fb0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:43.904492 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1326/2677] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 12:59:43 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 12:59:43.915762 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7126ba48-42c0-48eb-9379-ca7805a974d5 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:43.917458 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-7126ba48-42c0-48eb-9379-ca7805a974d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:43.973217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7126ba48-42c0-48eb-9379-ca7805a974d5 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:43.977534 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bcc894-eec8-4a31-b922-2e0149b903c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:43.978146 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcc894-eec8-4a31-b922-2e0149b903c8 service cinder] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:43.978424 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcc894-eec8-4a31-b922-2e0149b903c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:43.978693 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcc894-eec8-4a31-b922-2e0149b903c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:43.986907 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bcc894-eec8-4a31-b922-2e0149b903c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:43.988353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bcc894-eec8-4a31-b922-2e0149b903c8 service cinder] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:43.997656 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1352/2678] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:59:43 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 12:59:48.345878 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6ef4c-790d-4bec-a562-621a444fb0b6 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:48.347418 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-45e6ef4c-790d-4bec-a562-621a444fb0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:48.368062 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:59:48.407576 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6ef4c-790d-4bec-a562-621a444fb0b6 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:48.411157 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af4284-9f81-42a2-8ea1-5c1a8256053d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:48.411811 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af4284-9f81-42a2-8ea1-5c1a8256053d service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:48.412098 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af4284-9f81-42a2-8ea1-5c1a8256053d service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:48.412377 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af4284-9f81-42a2-8ea1-5c1a8256053d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:48.414113 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-07af4284-9f81-42a2-8ea1-5c1a8256053d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:48.471977 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07af4284-9f81-42a2-8ea1-5c1a8256053d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:48.473424 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07af4284-9f81-42a2-8ea1-5c1a8256053d service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:48.483008 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1327/2679] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:59:48 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:59:53.993702 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:DELETE=5,keystone:SELECT=64,keystone:INSERT=3 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 12:59:56.429828 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bcc894-eec8-4a31-b922-2e0149b903c8 service cinder] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:56.431248 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-a4bcc894-eec8-4a31-b922-2e0149b903c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:56.551184 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bcc894-eec8-4a31-b922-2e0149b903c8 service cinder] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:56.556545 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac9961-9be4-4153-83fb-120346ddde19 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec49a03aed774c439459883bd5442d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:56.557328 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac9961-9be4-4153-83fb-120346ddde19 service placement] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:56.557669 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac9961-9be4-4153-83fb-120346ddde19 service placement] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:56.558638 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac9961-9be4-4153-83fb-120346ddde19 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:56.560645 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-3bac9961-9be4-4153-83fb-120346ddde19 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:56.574900 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bac9961-9be4-4153-83fb-120346ddde19 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:56.578096 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bac9961-9be4-4153-83fb-120346ddde19 service placement] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:56.613424 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1353/2680] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:59:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:59:57.961259 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af4284-9f81-42a2-8ea1-5c1a8256053d service neutron] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:57.971164 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af4284-9f81-42a2-8ea1-5c1a8256053d service neutron] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:57.977977 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a3203-c4a6-4a71-8dad-950e7e6ec09d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:57.978602 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a3203-c4a6-4a71-8dad-950e7e6ec09d service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:57.979607 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a3203-c4a6-4a71-8dad-950e7e6ec09d service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:57.980405 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a3203-c4a6-4a71-8dad-950e7e6ec09d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:57.982285 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-790a3203-c4a6-4a71-8dad-950e7e6ec09d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:58.058453 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a3203-c4a6-4a71-8dad-950e7e6ec09d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 12:59:58.059985 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a3203-c4a6-4a71-8dad-950e7e6ec09d service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 12:59:58.069837 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1328/2681] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:59:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 12:59:59.974490 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac9961-9be4-4153-83fb-120346ddde19 service placement] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 12:59:59.976624 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-3bac9961-9be4-4153-83fb-120346ddde19 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 12:59:59.990025 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac9961-9be4-4153-83fb-120346ddde19 service placement] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 12:59:59.994320 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a67dee-f545-4ce5-a242-fd4786b5aec7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 12:59:59.994993 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a67dee-f545-4ce5-a242-fd4786b5aec7 service cinder] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 12:59:59.995277 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a67dee-f545-4ce5-a242-fd4786b5aec7 service cinder] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 12:59:59.996179 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a67dee-f545-4ce5-a242-fd4786b5aec7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 12:59:59.998826 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-75a67dee-f545-4ce5-a242-fd4786b5aec7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:00:00.071923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a67dee-f545-4ce5-a242-fd4786b5aec7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:00:00.073712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a67dee-f545-4ce5-a242-fd4786b5aec7 service cinder] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:00:00.098371 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1354/2682] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 12:59:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:00:01.292307 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a3203-c4a6-4a71-8dad-950e7e6ec09d service neutron] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:00:01.293907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-790a3203-c4a6-4a71-8dad-950e7e6ec09d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:00:01.310458 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a3203-c4a6-4a71-8dad-950e7e6ec09d service neutron] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:00:01.316446 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101933b8-9a74-4803-972a-24854f924fa1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec49a03aed774c439459883bd5442d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:00:01.317403 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101933b8-9a74-4803-972a-24854f924fa1 service placement] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:00:01.317786 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101933b8-9a74-4803-972a-24854f924fa1 service placement] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:00:01.318161 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101933b8-9a74-4803-972a-24854f924fa1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:00:01.320527 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-101933b8-9a74-4803-972a-24854f924fa1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:00:01.334276 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101933b8-9a74-4803-972a-24854f924fa1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:00:01.336188 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101933b8-9a74-4803-972a-24854f924fa1 service placement] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:00:01.360392 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1329/2683] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 13:00:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:00:01.376869 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a67dee-f545-4ce5-a242-fd4786b5aec7 service cinder] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:00:01.396852 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a67dee-f545-4ce5-a242-fd4786b5aec7 service cinder] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:00:01.421103 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e48699c-c108-46c5-99e9-8be14ae25bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:00:01.421942 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e48699c-c108-46c5-99e9-8be14ae25bc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:00:01.422728 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e48699c-c108-46c5-99e9-8be14ae25bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:00:01.423401 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e48699c-c108-46c5-99e9-8be14ae25bc4 admin admin] PATH_INFO: `/v3/users/77243319bb26419b9431bc9baf8ee871` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:00:01.424892 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e48699c-c108-46c5-99e9-8be14ae25bc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77243319bb26419b9431bc9baf8ee871)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:00:01.426962 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e48699c-c108-46c5-99e9-8be14ae25bc4 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:00:01.483356 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-7e48699c-c108-46c5-99e9-8be14ae25bc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77243319bb26419b9431bc9baf8ee871'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:00:01.494320 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-7e48699c-c108-46c5-99e9-8be14ae25bc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77243319bb26419b9431bc9baf8ee871'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:00:01.494896 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-7e48699c-c108-46c5-99e9-8be14ae25bc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77243319bb26419b9431bc9baf8ee871'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:00:01.505451 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-7e48699c-c108-46c5-99e9-8be14ae25bc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77243319bb26419b9431bc9baf8ee871'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:00:01.506932 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:00:01.526388 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-7e48699c-c108-46c5-99e9-8be14ae25bc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77243319bb26419b9431bc9baf8ee871'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:00:01.534337 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1355/2684] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 13:00:01 2022] DELETE /identity/v3/users/77243319bb26419b9431bc9baf8ee871 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:00:01.546128 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67065e46-a6ed-40c0-9419-212b27500663 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:00:01.546631 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67065e46-a6ed-40c0-9419-212b27500663 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:00:01.547004 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67065e46-a6ed-40c0-9419-212b27500663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:00:01.570446 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-67065e46-a6ed-40c0-9419-212b27500663 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:00:01.571831 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-67065e46-a6ed-40c0-9419-212b27500663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:00:01.635213 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:00:01.649253 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1330/2685] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 13:00:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:00:01.659476 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e48699c-c108-46c5-99e9-8be14ae25bc4 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:00:01.661181 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-7e48699c-c108-46c5-99e9-8be14ae25bc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:00:01.722155 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e48699c-c108-46c5-99e9-8be14ae25bc4 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:00:01.725819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8983ba0-b9dd-4b5f-babf-93765688e5cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:00:01.726444 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8983ba0-b9dd-4b5f-babf-93765688e5cb service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:00:01.726719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8983ba0-b9dd-4b5f-babf-93765688e5cb service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:00:01.726990 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8983ba0-b9dd-4b5f-babf-93765688e5cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:00:01.737085 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8983ba0-b9dd-4b5f-babf-93765688e5cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:00:01.738649 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8983ba0-b9dd-4b5f-babf-93765688e5cb service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:00:01.749973 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1356/2686] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 13:00:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:00:02.035862 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67065e46-a6ed-40c0-9419-212b27500663 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:00:02.037515 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-67065e46-a6ed-40c0-9419-212b27500663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:00:02.066519 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67065e46-a6ed-40c0-9419-212b27500663 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:00:02.070520 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2e790f-eef9-4265-8d2a-eb6d79264079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:00:02.071363 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e790f-eef9-4265-8d2a-eb6d79264079 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:00:02.071739 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e790f-eef9-4265-8d2a-eb6d79264079 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:00:02.072025 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e790f-eef9-4265-8d2a-eb6d79264079 admin admin] PATH_INFO: `/v3/projects/f47ad327b2a14f0fa121208b521e47cc` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:00:02.073324 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2e790f-eef9-4265-8d2a-eb6d79264079 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f47ad327b2a14f0fa121208b521e47cc)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:00:02.075598 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2e790f-eef9-4265-8d2a-eb6d79264079 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:00:02.126645 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-0a2e790f-eef9-4265-8d2a-eb6d79264079 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f47ad327b2a14f0fa121208b521e47cc', 'request_id': 'req-0a2e790f-eef9-4265-8d2a-eb6d79264079'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:00:02.133330 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-0a2e790f-eef9-4265-8d2a-eb6d79264079 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f47ad327b2a14f0fa121208b521e47cc', 'request_id': 'req-0a2e790f-eef9-4265-8d2a-eb6d79264079'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:00:02.142032 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-0a2e790f-eef9-4265-8d2a-eb6d79264079 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f47ad327b2a14f0fa121208b521e47cc', 'request_id': 'req-0a2e790f-eef9-4265-8d2a-eb6d79264079'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:00:02.154227 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-0a2e790f-eef9-4265-8d2a-eb6d79264079 admin admin] The token cache is being invalidate because project f47ad327b2a14f0fa121208b521e47cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:00:02.154520 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-0a2e790f-eef9-4265-8d2a-eb6d79264079 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:00:02.155163 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-0a2e790f-eef9-4265-8d2a-eb6d79264079 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:00:02.156753 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1331/2687] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 13:00:02 2022] DELETE /identity/v3/projects/f47ad327b2a14f0fa121208b521e47cc => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:00:02.161657 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91572d0-f4ea-4782-886b-f1fdddfe8c18 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:00:02.162011 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91572d0-f4ea-4782-886b-f1fdddfe8c18 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:00:02.163440 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91572d0-f4ea-4782-886b-f1fdddfe8c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:00:02.185536 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-d91572d0-f4ea-4782-886b-f1fdddfe8c18 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:00:02.186797 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-d91572d0-f4ea-4782-886b-f1fdddfe8c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:00:02.250363 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1357/2688] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 13:00:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:00:02.260466 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de9760-f54e-47ba-b130-3781611b3fe3 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:00:02.261047 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de9760-f54e-47ba-b130-3781611b3fe3 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:00:02.261445 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de9760-f54e-47ba-b130-3781611b3fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:00:02.283757 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-62de9760-f54e-47ba-b130-3781611b3fe3 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:00:02.284905 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-62de9760-f54e-47ba-b130-3781611b3fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:00:02.293656 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1332/2689] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 13:00:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:00:02.298053 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91572d0-f4ea-4782-886b-f1fdddfe8c18 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:00:02.308096 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91572d0-f4ea-4782-886b-f1fdddfe8c18 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:00:02.312715 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df45049-4cfe-4c35-bed5-a1ae41155e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:00:02.313323 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df45049-4cfe-4c35-bed5-a1ae41155e10 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:00:02.313615 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df45049-4cfe-4c35-bed5-a1ae41155e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:00:02.313877 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df45049-4cfe-4c35-bed5-a1ae41155e10 admin admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:00:02.314573 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df45049-4cfe-4c35-bed5-a1ae41155e10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:00:02.314884 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df45049-4cfe-4c35-bed5-a1ae41155e10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:00:02.316540 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df45049-4cfe-4c35-bed5-a1ae41155e10 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:00:02.332018 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1358/2690] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 13:00:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:00:02.336800 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de9760-f54e-47ba-b130-3781611b3fe3 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:00:02.348574 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de9760-f54e-47ba-b130-3781611b3fe3 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:00:02.353304 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1badad-c573-44e1-91e3-e10671064ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:00:02.353917 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1badad-c573-44e1-91e3-e10671064ffe admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:00:02.354196 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1badad-c573-44e1-91e3-e10671064ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:00:02.354472 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1badad-c573-44e1-91e3-e10671064ffe admin admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:00:02.355169 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1badad-c573-44e1-91e3-e10671064ffe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:00:02.357169 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1badad-c573-44e1-91e3-e10671064ffe admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:00:02.384756 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1333/2691] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 13:00:02 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:00:02.389563 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df45049-4cfe-4c35-bed5-a1ae41155e10 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:00:02.399511 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df45049-4cfe-4c35-bed5-a1ae41155e10 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:00:02.458712 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b792cbb-1fed-424d-8e96-0fc31d3d9048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:00:02.459332 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b792cbb-1fed-424d-8e96-0fc31d3d9048 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:00:02.459613 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b792cbb-1fed-424d-8e96-0fc31d3d9048 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:00:02.459917 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b792cbb-1fed-424d-8e96-0fc31d3d9048 admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:00:02.460597 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b792cbb-1fed-424d-8e96-0fc31d3d9048 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:00:02.462512 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b792cbb-1fed-424d-8e96-0fc31d3d9048 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:00:02.480620 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:00:02.489645 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1359/2692] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 13:00:02 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:00:02.490904 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1badad-c573-44e1-91e3-e10671064ffe admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:00:02.501252 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1badad-c573-44e1-91e3-e10671064ffe admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:00:02.507050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bf63bb-6f4c-4b09-9258-62b21107e769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:00:02.507660 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf63bb-6f4c-4b09-9258-62b21107e769 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:00:02.508204 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf63bb-6f4c-4b09-9258-62b21107e769 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:00:02.508475 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf63bb-6f4c-4b09-9258-62b21107e769 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:00:02.509170 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bf63bb-6f4c-4b09-9258-62b21107e769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:00:02.510765 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bf63bb-6f4c-4b09-9258-62b21107e769 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:00:02.522498 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1334/2693] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 13:00:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:00:02.526364 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b792cbb-1fed-424d-8e96-0fc31d3d9048 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:00:02.537166 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b792cbb-1fed-424d-8e96-0fc31d3d9048 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:00:02.542026 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbd94c4-8c4c-4b5e-9871-9b21c075e04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:00:02.542713 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd94c4-8c4c-4b5e-9871-9b21c075e04f admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:00:02.542994 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd94c4-8c4c-4b5e-9871-9b21c075e04f admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:00:02.543271 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd94c4-8c4c-4b5e-9871-9b21c075e04f admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:00:02.543995 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbd94c4-8c4c-4b5e-9871-9b21c075e04f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:00:02.545599 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbd94c4-8c4c-4b5e-9871-9b21c075e04f admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:00:02.556637 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1360/2694] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 13:00:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 13:00:02.560809 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bf63bb-6f4c-4b09-9258-62b21107e769 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:00:02.572461 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bf63bb-6f4c-4b09-9258-62b21107e769 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:00:02.578518 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c465b1-c023-4431-a704-5ee05214bbee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:00:02.579221 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c465b1-c023-4431-a704-5ee05214bbee admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:00:02.579505 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c465b1-c023-4431-a704-5ee05214bbee admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:00:02.579830 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c465b1-c023-4431-a704-5ee05214bbee admin admin] PATH_INFO: `/v3/projects/2ef477e2598b4c7bbd40c06ccd26112b/users/68818cd15fad466cb7cb5584212fe2f9/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:00:02.598906 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c465b1-c023-4431-a704-5ee05214bbee admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ef477e2598b4c7bbd40c06ccd26112b, user_id=68818cd15fad466cb7cb5584212fe2f9, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:00:02.601647 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c465b1-c023-4431-a704-5ee05214bbee admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:00:02.615852 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1335/2695] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 13:00:02 2022] PUT /identity/v3/projects/2ef477e2598b4c7bbd40c06ccd26112b/users/68818cd15fad466cb7cb5584212fe2f9/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:00:02.621858 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f15f63-fd60-469e-b786-796f660b54be None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:00:02.624411 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f15f63-fd60-469e-b786-796f660b54be None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:00:02.624411 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f15f63-fd60-469e-b786-796f660b54be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:00:02.644374 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-36f15f63-fd60-469e-b786-796f660b54be None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:00:02.645603 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-36f15f63-fd60-469e-b786-796f660b54be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:00:02.718482 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1361/2696] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 13:00:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:00:02.757572 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c465b1-c023-4431-a704-5ee05214bbee admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:00:02.759604 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-24c465b1-c023-4431-a704-5ee05214bbee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:00:02.832058 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c465b1-c023-4431-a704-5ee05214bbee admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:00:02.837026 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f569d198-80ab-4045-b70c-a15a812c8da0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:00:02.837656 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f569d198-80ab-4045-b70c-a15a812c8da0 service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:00:02.837933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f569d198-80ab-4045-b70c-a15a812c8da0 service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:00:02.838903 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f569d198-80ab-4045-b70c-a15a812c8da0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:00:02.849544 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f569d198-80ab-4045-b70c-a15a812c8da0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:00:02.851000 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f569d198-80ab-4045-b70c-a15a812c8da0 service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:00:02.861349 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1336/2697] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 13:00:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:00:05.784872 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f15f63-fd60-469e-b786-796f660b54be None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:00:05.787174 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-36f15f63-fd60-469e-b786-796f660b54be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:00:05.822791 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f15f63-fd60-469e-b786-796f660b54be None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:00:05.833295 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a5e0a3-4093-4306-b06d-010e17651e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:00:05.836801 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5e0a3-4093-4306-b06d-010e17651e8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:00:05.837101 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5e0a3-4093-4306-b06d-010e17651e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:00:05.837385 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5e0a3-4093-4306-b06d-010e17651e8b admin admin] PATH_INFO: `/v3/users/b6c556056d054689ab7b100f02404d3f` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:00:05.844935 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a5e0a3-4093-4306-b06d-010e17651e8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6c556056d054689ab7b100f02404d3f)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:00:05.847487 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a5e0a3-4093-4306-b06d-010e17651e8b admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:00:05.955153 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-48a5e0a3-4093-4306-b06d-010e17651e8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6c556056d054689ab7b100f02404d3f'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:00:05.969091 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-48a5e0a3-4093-4306-b06d-010e17651e8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6c556056d054689ab7b100f02404d3f'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:00:05.970695 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-48a5e0a3-4093-4306-b06d-010e17651e8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6c556056d054689ab7b100f02404d3f'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:00:05.987192 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-48a5e0a3-4093-4306-b06d-010e17651e8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6c556056d054689ab7b100f02404d3f'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:00:06.012080 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-48a5e0a3-4093-4306-b06d-010e17651e8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6c556056d054689ab7b100f02404d3f'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:00:06.037955 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1362/2698] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 13:00:05 2022] DELETE /identity/v3/users/b6c556056d054689ab7b100f02404d3f => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:00:06.320050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f569d198-80ab-4045-b70c-a15a812c8da0 service neutron] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:00:06.337491 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f569d198-80ab-4045-b70c-a15a812c8da0 service neutron] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:00:06.369581 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:00:06.438024 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95cc857-c718-4942-828f-3993e37f861d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:00:06.439930 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95cc857-c718-4942-828f-3993e37f861d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:00:06.440421 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95cc857-c718-4942-828f-3993e37f861d admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:00:06.440889 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95cc857-c718-4942-828f-3993e37f861d admin admin] PATH_INFO: `/v3/projects/b8383c3af34e46f2922ffb7f6d4963af` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:00:06.442662 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95cc857-c718-4942-828f-3993e37f861d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8383c3af34e46f2922ffb7f6d4963af)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:00:06.447352 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95cc857-c718-4942-828f-3993e37f861d admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:00:06.539706 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-a95cc857-c718-4942-828f-3993e37f861d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8383c3af34e46f2922ffb7f6d4963af', 'request_id': 'req-a95cc857-c718-4942-828f-3993e37f861d'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:00:06.557448 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-a95cc857-c718-4942-828f-3993e37f861d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8383c3af34e46f2922ffb7f6d4963af', 'request_id': 'req-a95cc857-c718-4942-828f-3993e37f861d'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:00:06.567352 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-a95cc857-c718-4942-828f-3993e37f861d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8383c3af34e46f2922ffb7f6d4963af', 'request_id': 'req-a95cc857-c718-4942-828f-3993e37f861d'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:00:06.582435 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-a95cc857-c718-4942-828f-3993e37f861d admin admin] The token cache is being invalidate because project b8383c3af34e46f2922ffb7f6d4963af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:00:06.584638 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-a95cc857-c718-4942-828f-3993e37f861d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:00:06.585394 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-a95cc857-c718-4942-828f-3993e37f861d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:00:06.587677 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1337/2699] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 13:00:06 2022] DELETE /identity/v3/projects/b8383c3af34e46f2922ffb7f6d4963af => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:00:06.591141 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a5e0a3-4093-4306-b06d-010e17651e8b admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:00:06.593213 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-48a5e0a3-4093-4306-b06d-010e17651e8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:00:06.614384 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a5e0a3-4093-4306-b06d-010e17651e8b admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:00:06.689397 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b94fa-c965-4160-94e3-065e60c79385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:00:06.690196 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b94fa-c965-4160-94e3-065e60c79385 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:00:06.690347 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b94fa-c965-4160-94e3-065e60c79385 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:00:06.690629 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b94fa-c965-4160-94e3-065e60c79385 admin admin] PATH_INFO: `/v3/users/8639ac065f9d4ae5b3a9bd771bfbd045` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:00:06.691923 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8b94fa-c965-4160-94e3-065e60c79385 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8639ac065f9d4ae5b3a9bd771bfbd045)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:00:06.693911 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8b94fa-c965-4160-94e3-065e60c79385 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:00:06.696881 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:00:06.770146 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-da8b94fa-c965-4160-94e3-065e60c79385 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8639ac065f9d4ae5b3a9bd771bfbd045'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:00:06.780763 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-da8b94fa-c965-4160-94e3-065e60c79385 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8639ac065f9d4ae5b3a9bd771bfbd045'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:00:06.781442 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-da8b94fa-c965-4160-94e3-065e60c79385 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8639ac065f9d4ae5b3a9bd771bfbd045'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:00:06.790539 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-da8b94fa-c965-4160-94e3-065e60c79385 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8639ac065f9d4ae5b3a9bd771bfbd045'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:00:06.804552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-da8b94fa-c965-4160-94e3-065e60c79385 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8639ac065f9d4ae5b3a9bd771bfbd045'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:00:06.827788 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1363/2700] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 13:00:06 2022] DELETE /identity/v3/users/8639ac065f9d4ae5b3a9bd771bfbd045 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:00:07.035911 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95cc857-c718-4942-828f-3993e37f861d admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:00:07.037683 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-a95cc857-c718-4942-828f-3993e37f861d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:00:07.056631 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95cc857-c718-4942-828f-3993e37f861d admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:00:07.128378 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094f991a-c583-4fe0-9bc9-f54f43369b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:00:07.129894 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f991a-c583-4fe0-9bc9-f54f43369b15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:00:07.130196 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f991a-c583-4fe0-9bc9-f54f43369b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:00:07.130477 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f991a-c583-4fe0-9bc9-f54f43369b15 admin admin] PATH_INFO: `/v3/projects/fd4ed91d9355467b98e8bb224ec9d05e` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:00:07.132299 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094f991a-c583-4fe0-9bc9-f54f43369b15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd4ed91d9355467b98e8bb224ec9d05e)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:00:07.134610 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094f991a-c583-4fe0-9bc9-f54f43369b15 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:00:07.193484 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:00:07.196092 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-094f991a-c583-4fe0-9bc9-f54f43369b15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd4ed91d9355467b98e8bb224ec9d05e', 'request_id': 'req-094f991a-c583-4fe0-9bc9-f54f43369b15'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:00:07.206275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-094f991a-c583-4fe0-9bc9-f54f43369b15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd4ed91d9355467b98e8bb224ec9d05e', 'request_id': 'req-094f991a-c583-4fe0-9bc9-f54f43369b15'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:00:07.217008 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-094f991a-c583-4fe0-9bc9-f54f43369b15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd4ed91d9355467b98e8bb224ec9d05e', 'request_id': 'req-094f991a-c583-4fe0-9bc9-f54f43369b15'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:00:07.232721 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-094f991a-c583-4fe0-9bc9-f54f43369b15 admin admin] The token cache is being invalidate because project fd4ed91d9355467b98e8bb224ec9d05e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:00:07.233105 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-094f991a-c583-4fe0-9bc9-f54f43369b15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:00:07.233815 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-094f991a-c583-4fe0-9bc9-f54f43369b15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:00:07.238293 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1338/2701] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 13:00:07 2022] DELETE /identity/v3/projects/fd4ed91d9355467b98e8bb224ec9d05e => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:00:07.530632 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6b850-4362-4bbf-8038-b2f94a4ca1e5 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:00:07.531260 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6b850-4362-4bbf-8038-b2f94a4ca1e5 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:00:07.531829 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6b850-4362-4bbf-8038-b2f94a4ca1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:00:07.568804 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-fba6b850-4362-4bbf-8038-b2f94a4ca1e5 None None] MFA Rules not processed for user `94efba410653451da2e9977d746633f6`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:00:07.570033 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-fba6b850-4362-4bbf-8038-b2f94a4ca1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:00:07.659457 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1364/2702] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 13:00:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:00:08.421874 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be882d31-6ce4-4faa-a2dd-caa49b33feec None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:00:08.422459 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be882d31-6ce4-4faa-a2dd-caa49b33feec None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:00:08.422459 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be882d31-6ce4-4faa-a2dd-caa49b33feec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:00:08.458565 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-be882d31-6ce4-4faa-a2dd-caa49b33feec None None] MFA Rules not processed for user `68818cd15fad466cb7cb5584212fe2f9`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:00:08.459943 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-be882d31-6ce4-4faa-a2dd-caa49b33feec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:00:08.574627 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1339/2703] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 13:00:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 13:00:08.580916 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba6b850-4362-4bbf-8038-b2f94a4ca1e5 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:00:08.584611 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-fba6b850-4362-4bbf-8038-b2f94a4ca1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:00:08.701384 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba6b850-4362-4bbf-8038-b2f94a4ca1e5 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:00:08.710521 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd595ce-6116-48cc-be00-72e8228e2508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:00:08.711171 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd595ce-6116-48cc-be00-72e8228e2508 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:00:08.711457 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd595ce-6116-48cc-be00-72e8228e2508 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:00:08.711771 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd595ce-6116-48cc-be00-72e8228e2508 admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:00:08.712516 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd595ce-6116-48cc-be00-72e8228e2508 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:00:08.714584 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd595ce-6116-48cc-be00-72e8228e2508 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:00:08.763386 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1365/2704] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 13:00:08 2022] POST /identity/v3/projects => generated 385 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:00:08.769209 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be882d31-6ce4-4faa-a2dd-caa49b33feec None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:00:08.787329 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be882d31-6ce4-4faa-a2dd-caa49b33feec None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:00:08.874211 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11815f83-dc48-4745-8a41-5a0291e261b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:00:08.874848 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11815f83-dc48-4745-8a41-5a0291e261b8 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:00:08.875129 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11815f83-dc48-4745-8a41-5a0291e261b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:00:08.875399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11815f83-dc48-4745-8a41-5a0291e261b8 admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:00:08.876404 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11815f83-dc48-4745-8a41-5a0291e261b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:00:08.878984 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11815f83-dc48-4745-8a41-5a0291e261b8 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:00:08.929089 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1340/2705] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 13:00:08 2022] POST /identity/v3/users => generated 299 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:00:08.930612 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd595ce-6116-48cc-be00-72e8228e2508 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:00:08.951465 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd595ce-6116-48cc-be00-72e8228e2508 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:00:08.975041 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e493276-8f54-421c-ad4c-0ed12959c479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:00:08.975987 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e493276-8f54-421c-ad4c-0ed12959c479 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:00:08.976469 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e493276-8f54-421c-ad4c-0ed12959c479 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:00:08.976931 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e493276-8f54-421c-ad4c-0ed12959c479 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:00:08.977795 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e493276-8f54-421c-ad4c-0ed12959c479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:00:08.985760 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e493276-8f54-421c-ad4c-0ed12959c479 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:00:08.995998 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:00:09.003273 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1366/2706] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 13:00:08 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:00:09.007621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11815f83-dc48-4745-8a41-5a0291e261b8 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:00:09.019596 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11815f83-dc48-4745-8a41-5a0291e261b8 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:00:09.024868 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7029fff0-4cb8-4561-b1f5-c56d57907265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:00:09.025733 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7029fff0-4cb8-4561-b1f5-c56d57907265 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:00:09.026017 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7029fff0-4cb8-4561-b1f5-c56d57907265 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:00:09.026297 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7029fff0-4cb8-4561-b1f5-c56d57907265 admin admin] PATH_INFO: `/v3/projects/48549592401f40a4a75d41d0a274cb8e/users/8c94c2ead3be491889350f563a9f808b/roles/0e78444e43c744668178971af4087ca0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:00:09.062408 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7029fff0-4cb8-4561-b1f5-c56d57907265 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48549592401f40a4a75d41d0a274cb8e, user_id=8c94c2ead3be491889350f563a9f808b, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:00:09.065165 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7029fff0-4cb8-4561-b1f5-c56d57907265 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:00:09.097594 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1341/2707] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 13:00:09 2022] PUT /identity/v3/projects/48549592401f40a4a75d41d0a274cb8e/users/8c94c2ead3be491889350f563a9f808b/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:00:09.099521 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e493276-8f54-421c-ad4c-0ed12959c479 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:00:09.123336 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e493276-8f54-421c-ad4c-0ed12959c479 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:00:09.243344 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac5f50-2ba2-463e-990f-8f9b3bb703c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:00:09.247488 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac5f50-2ba2-463e-990f-8f9b3bb703c5 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:00:09.247488 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac5f50-2ba2-463e-990f-8f9b3bb703c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:00:09.247488 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac5f50-2ba2-463e-990f-8f9b3bb703c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:00:09.247786 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac5f50-2ba2-463e-990f-8f9b3bb703c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:00:09.254098 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac5f50-2ba2-463e-990f-8f9b3bb703c5 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:00:09.267892 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1367/2708] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 13:00:09 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:00:09.271924 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7029fff0-4cb8-4561-b1f5-c56d57907265 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:00:09.284792 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7029fff0-4cb8-4561-b1f5-c56d57907265 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:00:09.290675 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b3c669-acee-43d2-be0d-ce07c8569ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:00:09.291589 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3c669-acee-43d2-be0d-ce07c8569ced admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:00:09.292204 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3c669-acee-43d2-be0d-ce07c8569ced admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:00:09.292668 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3c669-acee-43d2-be0d-ce07c8569ced admin admin] PATH_INFO: `/v3/projects/48549592401f40a4a75d41d0a274cb8e/users/8c94c2ead3be491889350f563a9f808b/roles/0e78444e43c744668178971af4087ca0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:00:09.295388 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b3c669-acee-43d2-be0d-ce07c8569ced admin admin] RBAC: Authorizing `identity:create_grant(project_id=48549592401f40a4a75d41d0a274cb8e, user_id=8c94c2ead3be491889350f563a9f808b, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:00:09.298956 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b3c669-acee-43d2-be0d-ce07c8569ced admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:00:09.312625 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1342/2709] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 13:00:09 2022] PUT /identity/v3/projects/48549592401f40a4a75d41d0a274cb8e/users/8c94c2ead3be491889350f563a9f808b/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 13:00:15.812825 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97198a43-aef2-45e2-9e28-f9fde4c6685f None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:00:15.813409 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97198a43-aef2-45e2-9e28-f9fde4c6685f None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:00:15.813409 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97198a43-aef2-45e2-9e28-f9fde4c6685f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:00:15.839969 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-97198a43-aef2-45e2-9e28-f9fde4c6685f None None] MFA Rules not processed for user `d480ea8c18634b88b3248a80422aa798`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:00:15.842098 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-97198a43-aef2-45e2-9e28-f9fde4c6685f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:00:15.957475 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1368/2710] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 13:00:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:00:17.003080 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3009980-70cf-4fd3-bea1-e2818d17a409 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:00:17.003618 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3009980-70cf-4fd3-bea1-e2818d17a409 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:00:17.003618 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3009980-70cf-4fd3-bea1-e2818d17a409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:00:17.035682 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-d3009980-70cf-4fd3-bea1-e2818d17a409 None None] MFA Rules not processed for user `8c94c2ead3be491889350f563a9f808b`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:00:17.037304 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-d3009980-70cf-4fd3-bea1-e2818d17a409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:00:17.044498 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=91,keystone:INSERT=6,keystone:DELETE=3 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:00:17.145635 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1343/2711] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 13:00:17 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:00:17.158992 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97198a43-aef2-45e2-9e28-f9fde4c6685f None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:00:17.162132 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-97198a43-aef2-45e2-9e28-f9fde4c6685f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:00:17.232242 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97198a43-aef2-45e2-9e28-f9fde4c6685f None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:00:17.236304 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8d2666-835a-49c8-be8b-12ec5797b67b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2b5853838dc40db89c46038c0606107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:00:17.236930 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d2666-835a-49c8-be8b-12ec5797b67b service glance] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:00:17.237207 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d2666-835a-49c8-be8b-12ec5797b67b service glance] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:00:17.237477 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d2666-835a-49c8-be8b-12ec5797b67b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:00:17.297160 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:00:17.300410 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8d2666-835a-49c8-be8b-12ec5797b67b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:00:17.302284 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8d2666-835a-49c8-be8b-12ec5797b67b service glance] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:00:17.319127 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1369/2712] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 13:00:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:00:17.472453 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175fb265-6141-4432-85d9-63a0dfba2c7f None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:00:17.472744 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175fb265-6141-4432-85d9-63a0dfba2c7f None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:00:17.473069 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175fb265-6141-4432-85d9-63a0dfba2c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:00:17.500889 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-175fb265-6141-4432-85d9-63a0dfba2c7f None None] MFA Rules not processed for user `30fc532f464843a3ad35e4d3c387bb3a`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:00:17.502126 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-175fb265-6141-4432-85d9-63a0dfba2c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:00:17.579610 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1344/2713] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Jul 26 13:00:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:00:17.592784 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8d2666-835a-49c8-be8b-12ec5797b67b service glance] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:00:17.594794 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-8f8d2666-835a-49c8-be8b-12ec5797b67b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:00:17.696541 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8d2666-835a-49c8-be8b-12ec5797b67b service glance] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:00:17.703035 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329aeeaf-ec04-4055-9518-aff183affcad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd2483f74d7d48b1a7b8ffe32f63abde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:00:17.703684 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329aeeaf-ec04-4055-9518-aff183affcad service swift] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:00:17.703974 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329aeeaf-ec04-4055-9518-aff183affcad service swift] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:00:17.704252 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329aeeaf-ec04-4055-9518-aff183affcad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:00:17.721284 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329aeeaf-ec04-4055-9518-aff183affcad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:00:17.723159 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329aeeaf-ec04-4055-9518-aff183affcad service swift] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:00:17.744928 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1370/2714] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Jul 26 13:00:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 13:00:26.219048 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175fb265-6141-4432-85d9-63a0dfba2c7f None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:00:26.220571 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-175fb265-6141-4432-85d9-63a0dfba2c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:00:26.291475 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175fb265-6141-4432-85d9-63a0dfba2c7f None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:00:26.297691 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b2048-b030-488a-9215-0bd9376e87cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:00:26.298399 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b2048-b030-488a-9215-0bd9376e87cd service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:00:26.298750 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b2048-b030-488a-9215-0bd9376e87cd service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:00:26.299115 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b2048-b030-488a-9215-0bd9376e87cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:00:26.301169 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-6a8b2048-b030-488a-9215-0bd9376e87cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:00:26.373528 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8b2048-b030-488a-9215-0bd9376e87cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:00:26.375093 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8b2048-b030-488a-9215-0bd9376e87cd service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:00:26.379523 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:00:26.393005 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1345/2715] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 13:00:26 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:00:27.732815 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=34 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:00:28.595436 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718cc1b-2ba4-4cc2-aeeb-f590e115fa0b None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:00:28.596161 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718cc1b-2ba4-4cc2-aeeb-f590e115fa0b None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:00:28.596161 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718cc1b-2ba4-4cc2-aeeb-f590e115fa0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:00:28.621222 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-3718cc1b-2ba4-4cc2-aeeb-f590e115fa0b None None] MFA Rules not processed for user `68818cd15fad466cb7cb5584212fe2f9`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:00:28.622514 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-3718cc1b-2ba4-4cc2-aeeb-f590e115fa0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:00:28.637708 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1371/2716] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 13:00:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:00:30.565191 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d775f2-073c-4bae-942c-bca0626060d1 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:00:30.566258 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d775f2-073c-4bae-942c-bca0626060d1 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:00:30.566708 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d775f2-073c-4bae-942c-bca0626060d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:00:30.598428 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-48d775f2-073c-4bae-942c-bca0626060d1 None None] MFA Rules not processed for user `30fc532f464843a3ad35e4d3c387bb3a`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:00:30.599684 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-48d775f2-073c-4bae-942c-bca0626060d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:00:30.607172 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1346/2717] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Jul 26 13:00:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:00:30.624033 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3718cc1b-2ba4-4cc2-aeeb-f590e115fa0b None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:00:30.642095 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3718cc1b-2ba4-4cc2-aeeb-f590e115fa0b None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:00:30.647578 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2a1d8-56a3-43d7-94ed-f23b96110a02 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd2483f74d7d48b1a7b8ffe32f63abde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:00:30.648247 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2a1d8-56a3-43d7-94ed-f23b96110a02 service swift] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:00:30.648527 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2a1d8-56a3-43d7-94ed-f23b96110a02 service swift] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:00:30.648794 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2a1d8-56a3-43d7-94ed-f23b96110a02 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:00:30.660400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d2a1d8-56a3-43d7-94ed-f23b96110a02 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:00:30.662537 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d2a1d8-56a3-43d7-94ed-f23b96110a02 service swift] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:00:30.681835 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1372/2718] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Jul 26 13:00:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 13:00:40.592532 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=4 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:00:40.669203 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=9 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:00:57.468163 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d775f2-073c-4bae-942c-bca0626060d1 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:00:57.477745 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d775f2-073c-4bae-942c-bca0626060d1 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:00:57.481458 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf8d370-634a-43cb-bdb8-27d727470722 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:00:57.482058 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8d370-634a-43cb-bdb8-27d727470722 service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:00:57.482342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8d370-634a-43cb-bdb8-27d727470722 service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:00:57.482627 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8d370-634a-43cb-bdb8-27d727470722 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:00:57.491531 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf8d370-634a-43cb-bdb8-27d727470722 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:00:57.492985 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf8d370-634a-43cb-bdb8-27d727470722 service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:00:57.502446 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1347/2719] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 13:00:57 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:01:07.498249 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=6 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:01:17.799182 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2a1d8-56a3-43d7-94ed-f23b96110a02 service swift] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:17.800574 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-e4d2a1d8-56a3-43d7-94ed-f23b96110a02 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:17.855259 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2a1d8-56a3-43d7-94ed-f23b96110a02 service swift] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:17.858849 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9b494b-b640-47db-80e7-1c643a42d95d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:17.859449 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b494b-b640-47db-80e7-1c643a42d95d service cinder] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:17.859753 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b494b-b640-47db-80e7-1c643a42d95d service cinder] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:17.860025 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b494b-b640-47db-80e7-1c643a42d95d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:17.861726 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-ff9b494b-b640-47db-80e7-1c643a42d95d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:17.944075 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9b494b-b640-47db-80e7-1c643a42d95d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:17.945701 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9b494b-b640-47db-80e7-1c643a42d95d service cinder] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:17.966301 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1373/2720] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 13:01:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:01:26.013807 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf8d370-634a-43cb-bdb8-27d727470722 service neutron] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:26.015306 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-9cf8d370-634a-43cb-bdb8-27d727470722 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:26.028394 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf8d370-634a-43cb-bdb8-27d727470722 service neutron] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:26.032134 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9fa8ff-ca02-4266-b419-c7cde5654ba6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:26.032734 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9fa8ff-ca02-4266-b419-c7cde5654ba6 service cinder] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:26.033014 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9fa8ff-ca02-4266-b419-c7cde5654ba6 service cinder] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:26.033282 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9fa8ff-ca02-4266-b419-c7cde5654ba6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:26.035021 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-de9fa8ff-ca02-4266-b419-c7cde5654ba6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:26.049057 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9fa8ff-ca02-4266-b419-c7cde5654ba6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:26.050530 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9fa8ff-ca02-4266-b419-c7cde5654ba6 service cinder] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:26.062344 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1348/2721] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 13:01:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 13:01:27.963743 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=63 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:01:36.056701 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=6 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:01:40.041620 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9b494b-b640-47db-80e7-1c643a42d95d service cinder] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:40.043104 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-ff9b494b-b640-47db-80e7-1c643a42d95d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:40.113322 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9b494b-b640-47db-80e7-1c643a42d95d service cinder] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:40.117937 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c52e0ae-fa32-4619-ba8e-c5575c29da44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:40.118573 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52e0ae-fa32-4619-ba8e-c5575c29da44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:40.118854 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52e0ae-fa32-4619-ba8e-c5575c29da44 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:40.119127 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52e0ae-fa32-4619-ba8e-c5575c29da44 admin admin] PATH_INFO: `/v3/users/d480ea8c18634b88b3248a80422aa798` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:40.120317 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c52e0ae-fa32-4619-ba8e-c5575c29da44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d480ea8c18634b88b3248a80422aa798)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:40.122237 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c52e0ae-fa32-4619-ba8e-c5575c29da44 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:40.172465 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-1c52e0ae-fa32-4619-ba8e-c5575c29da44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd480ea8c18634b88b3248a80422aa798'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:40.180961 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-1c52e0ae-fa32-4619-ba8e-c5575c29da44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd480ea8c18634b88b3248a80422aa798'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:40.181773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-1c52e0ae-fa32-4619-ba8e-c5575c29da44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd480ea8c18634b88b3248a80422aa798'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:40.188893 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-1c52e0ae-fa32-4619-ba8e-c5575c29da44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd480ea8c18634b88b3248a80422aa798'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:40.200449 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-1c52e0ae-fa32-4619-ba8e-c5575c29da44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd480ea8c18634b88b3248a80422aa798'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:40.206800 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1374/2722] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 13:01:40 2022] DELETE /identity/v3/users/d480ea8c18634b88b3248a80422aa798 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:01:40.374892 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9fa8ff-ca02-4266-b419-c7cde5654ba6 service cinder] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:40.391063 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9fa8ff-ca02-4266-b419-c7cde5654ba6 service cinder] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:40.449484 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7e6f35-2886-4431-976c-d95cb317f1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:40.450039 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e6f35-2886-4431-976c-d95cb317f1f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:40.450324 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e6f35-2886-4431-976c-d95cb317f1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:40.450603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e6f35-2886-4431-976c-d95cb317f1f3 admin admin] PATH_INFO: `/v3/projects/429715c1b71244bcbbf9f30190d25823` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:40.452297 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7e6f35-2886-4431-976c-d95cb317f1f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=429715c1b71244bcbbf9f30190d25823)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:40.454212 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7e6f35-2886-4431-976c-d95cb317f1f3 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:40.507453 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-3a7e6f35-2886-4431-976c-d95cb317f1f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '429715c1b71244bcbbf9f30190d25823', 'request_id': 'req-3a7e6f35-2886-4431-976c-d95cb317f1f3'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:40.514792 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-3a7e6f35-2886-4431-976c-d95cb317f1f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '429715c1b71244bcbbf9f30190d25823', 'request_id': 'req-3a7e6f35-2886-4431-976c-d95cb317f1f3'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:40.522769 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-3a7e6f35-2886-4431-976c-d95cb317f1f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '429715c1b71244bcbbf9f30190d25823', 'request_id': 'req-3a7e6f35-2886-4431-976c-d95cb317f1f3'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:40.533729 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-3a7e6f35-2886-4431-976c-d95cb317f1f3 admin admin] The token cache is being invalidate because project 429715c1b71244bcbbf9f30190d25823 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:01:40.534018 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-3a7e6f35-2886-4431-976c-d95cb317f1f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:40.534662 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-3a7e6f35-2886-4431-976c-d95cb317f1f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:40.536242 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1349/2723] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 13:01:40 2022] DELETE /identity/v3/projects/429715c1b71244bcbbf9f30190d25823 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 13:01:40.541297 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c4bb6-786d-47cd-8943-0cdd3985e21f None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:40.541590 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c4bb6-786d-47cd-8943-0cdd3985e21f None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:40.541861 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c4bb6-786d-47cd-8943-0cdd3985e21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:40.562499 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-fa1c4bb6-786d-47cd-8943-0cdd3985e21f None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:01:40.563719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-fa1c4bb6-786d-47cd-8943-0cdd3985e21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:40.622402 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1375/2724] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 13:01:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:01:40.630835 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1535207-27da-4ef1-bdba-b67acef5ab4d None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:40.631123 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1535207-27da-4ef1-bdba-b67acef5ab4d None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:40.631409 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1535207-27da-4ef1-bdba-b67acef5ab4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:40.652241 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-b1535207-27da-4ef1-bdba-b67acef5ab4d None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:01:40.653447 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-b1535207-27da-4ef1-bdba-b67acef5ab4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:40.661918 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1350/2725] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 13:01:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:01:40.666163 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1c4bb6-786d-47cd-8943-0cdd3985e21f None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:40.675905 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1c4bb6-786d-47cd-8943-0cdd3985e21f None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:40.680514 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee671f-e2b2-487d-884d-13d445e47ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:40.681111 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee671f-e2b2-487d-884d-13d445e47ac8 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:40.681390 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee671f-e2b2-487d-884d-13d445e47ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:40.681662 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee671f-e2b2-487d-884d-13d445e47ac8 admin admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:40.682357 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee671f-e2b2-487d-884d-13d445e47ac8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:01:40.682659 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee671f-e2b2-487d-884d-13d445e47ac8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:40.684306 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee671f-e2b2-487d-884d-13d445e47ac8 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:40.696286 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1376/2726] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 13:01:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:01:40.700664 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1535207-27da-4ef1-bdba-b67acef5ab4d None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:40.710624 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1535207-27da-4ef1-bdba-b67acef5ab4d None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:40.715421 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0093c06a-809c-4504-8234-dc129fe7486c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:40.716058 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0093c06a-809c-4504-8234-dc129fe7486c admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:40.716341 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0093c06a-809c-4504-8234-dc129fe7486c admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:40.716616 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0093c06a-809c-4504-8234-dc129fe7486c admin admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:40.717306 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0093c06a-809c-4504-8234-dc129fe7486c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:40.719214 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0093c06a-809c-4504-8234-dc129fe7486c admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:40.756181 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1351/2727] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 13:01:40 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:01:40.760811 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee671f-e2b2-487d-884d-13d445e47ac8 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:40.771376 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee671f-e2b2-487d-884d-13d445e47ac8 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:40.774877 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:01:40.839252 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7b3670-50d3-4239-bd55-54512ebec70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:40.840095 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b3670-50d3-4239-bd55-54512ebec70f admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:40.840373 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b3670-50d3-4239-bd55-54512ebec70f admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:40.840644 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b3670-50d3-4239-bd55-54512ebec70f admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:40.841312 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7b3670-50d3-4239-bd55-54512ebec70f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:40.843266 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7b3670-50d3-4239-bd55-54512ebec70f admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:40.865452 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1377/2728] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 13:01:40 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:01:40.869753 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0093c06a-809c-4504-8234-dc129fe7486c admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:40.880675 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0093c06a-809c-4504-8234-dc129fe7486c admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:40.885624 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc20352-c1b3-40ff-9cfc-ac93f1e4f9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:40.886221 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc20352-c1b3-40ff-9cfc-ac93f1e4f9a7 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:40.886502 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc20352-c1b3-40ff-9cfc-ac93f1e4f9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:40.886776 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc20352-c1b3-40ff-9cfc-ac93f1e4f9a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:40.887466 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc20352-c1b3-40ff-9cfc-ac93f1e4f9a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:40.889088 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc20352-c1b3-40ff-9cfc-ac93f1e4f9a7 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:40.898999 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1352/2729] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 13:01:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:01:40.903444 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7b3670-50d3-4239-bd55-54512ebec70f admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:40.913937 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7b3670-50d3-4239-bd55-54512ebec70f admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:40.918537 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a6350-7c15-42bd-9e27-ba3bd536d19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:40.919151 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a6350-7c15-42bd-9e27-ba3bd536d19c admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:40.919445 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a6350-7c15-42bd-9e27-ba3bd536d19c admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:40.919750 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a6350-7c15-42bd-9e27-ba3bd536d19c admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:40.920972 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4a6350-7c15-42bd-9e27-ba3bd536d19c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:40.922694 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4a6350-7c15-42bd-9e27-ba3bd536d19c admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:40.933582 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1378/2730] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 13:01:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:01:40.938424 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc20352-c1b3-40ff-9cfc-ac93f1e4f9a7 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:40.948904 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc20352-c1b3-40ff-9cfc-ac93f1e4f9a7 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:40.953951 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5719bbe-c798-4729-a6a1-a8424d36f974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:40.954649 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5719bbe-c798-4729-a6a1-a8424d36f974 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:40.954962 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5719bbe-c798-4729-a6a1-a8424d36f974 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:40.955319 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5719bbe-c798-4729-a6a1-a8424d36f974 admin admin] PATH_INFO: `/v3/projects/2ab311ac1d334258ae30db35c83fcc2f/users/b02926e72e3e4d7d959a24657de527d1/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:40.970293 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5719bbe-c798-4729-a6a1-a8424d36f974 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ab311ac1d334258ae30db35c83fcc2f, user_id=b02926e72e3e4d7d959a24657de527d1, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:40.972999 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5719bbe-c798-4729-a6a1-a8424d36f974 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:40.986308 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1353/2731] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 13:01:40 2022] PUT /identity/v3/projects/2ab311ac1d334258ae30db35c83fcc2f/users/b02926e72e3e4d7d959a24657de527d1/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:01:40.997026 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b33a9-dbd7-499a-a8d9-eefcbfc66533 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:40.997317 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b33a9-dbd7-499a-a8d9-eefcbfc66533 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:40.997591 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b33a9-dbd7-499a-a8d9-eefcbfc66533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:41.025617 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-786b33a9-dbd7-499a-a8d9-eefcbfc66533 None None] MFA Rules not processed for user `b02926e72e3e4d7d959a24657de527d1`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:01:41.026819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-786b33a9-dbd7-499a-a8d9-eefcbfc66533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:41.090561 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1379/2732] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 13:01:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:01:41.101564 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5719bbe-c798-4729-a6a1-a8424d36f974 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:41.103311 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-d5719bbe-c798-4729-a6a1-a8424d36f974 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:41.124959 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:01:41.182893 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5719bbe-c798-4729-a6a1-a8424d36f974 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:41.189217 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816fd11-d450-42b0-a452-be6b9f459520 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:41.189832 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816fd11-d450-42b0-a452-be6b9f459520 service nova] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:41.190111 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816fd11-d450-42b0-a452-be6b9f459520 service nova] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:41.190387 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816fd11-d450-42b0-a452-be6b9f459520 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:41.198092 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0816fd11-d450-42b0-a452-be6b9f459520 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:41.199695 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0816fd11-d450-42b0-a452-be6b9f459520 service nova] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:41.208547 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1354/2733] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 13:01:41 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 13:01:43.875904 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b33a9-dbd7-499a-a8d9-eefcbfc66533 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:43.936590 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b33a9-dbd7-499a-a8d9-eefcbfc66533 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:43.941183 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f654cb-4f1b-4c34-973f-2eb2c7f0f5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:43.941819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f654cb-4f1b-4c34-973f-2eb2c7f0f5e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:43.942099 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f654cb-4f1b-4c34-973f-2eb2c7f0f5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:43.942375 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f654cb-4f1b-4c34-973f-2eb2c7f0f5e1 admin admin] PATH_INFO: `/v3/users/b02926e72e3e4d7d959a24657de527d1` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:43.943529 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f654cb-4f1b-4c34-973f-2eb2c7f0f5e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b02926e72e3e4d7d959a24657de527d1)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:43.945489 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f654cb-4f1b-4c34-973f-2eb2c7f0f5e1 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:43.962151 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:01:43.996304 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-19f654cb-4f1b-4c34-973f-2eb2c7f0f5e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b02926e72e3e4d7d959a24657de527d1'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:44.005009 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-19f654cb-4f1b-4c34-973f-2eb2c7f0f5e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b02926e72e3e4d7d959a24657de527d1'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:44.005819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-19f654cb-4f1b-4c34-973f-2eb2c7f0f5e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b02926e72e3e4d7d959a24657de527d1'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:44.013996 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-19f654cb-4f1b-4c34-973f-2eb2c7f0f5e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b02926e72e3e4d7d959a24657de527d1'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:44.026322 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-19f654cb-4f1b-4c34-973f-2eb2c7f0f5e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b02926e72e3e4d7d959a24657de527d1'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:44.032233 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1380/2734] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 13:01:43 2022] DELETE /identity/v3/users/b02926e72e3e4d7d959a24657de527d1 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:01:44.036890 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87724d58-739c-4a10-86a2-a425436c2e50 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:44.037374 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87724d58-739c-4a10-86a2-a425436c2e50 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:44.037503 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87724d58-739c-4a10-86a2-a425436c2e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:44.056901 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-87724d58-739c-4a10-86a2-a425436c2e50 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:01:44.058109 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-87724d58-739c-4a10-86a2-a425436c2e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:44.117665 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1355/2735] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 13:01:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:01:44.127982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f654cb-4f1b-4c34-973f-2eb2c7f0f5e1 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:44.129413 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-19f654cb-4f1b-4c34-973f-2eb2c7f0f5e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:44.187765 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f654cb-4f1b-4c34-973f-2eb2c7f0f5e1 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:44.191442 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37947a4a-7c6d-499c-8d63-19c102068b9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:44.192095 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37947a4a-7c6d-499c-8d63-19c102068b9a service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:44.192379 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37947a4a-7c6d-499c-8d63-19c102068b9a service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:44.192653 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37947a4a-7c6d-499c-8d63-19c102068b9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:44.201886 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37947a4a-7c6d-499c-8d63-19c102068b9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:44.203313 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37947a4a-7c6d-499c-8d63-19c102068b9a service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:44.212915 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1381/2736] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 13:01:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 13:01:44.483032 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87724d58-739c-4a10-86a2-a425436c2e50 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:44.484686 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-87724d58-739c-4a10-86a2-a425436c2e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:44.496888 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87724d58-739c-4a10-86a2-a425436c2e50 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:44.501648 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aa3d4b-dbba-451a-935d-affa33603540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:44.502296 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa3d4b-dbba-451a-935d-affa33603540 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:44.502584 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa3d4b-dbba-451a-935d-affa33603540 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:44.502884 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa3d4b-dbba-451a-935d-affa33603540 admin admin] PATH_INFO: `/v3/projects/2ab311ac1d334258ae30db35c83fcc2f` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:44.504204 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aa3d4b-dbba-451a-935d-affa33603540 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ab311ac1d334258ae30db35c83fcc2f)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:44.506155 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aa3d4b-dbba-451a-935d-affa33603540 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:44.554143 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-d0aa3d4b-dbba-451a-935d-affa33603540 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ab311ac1d334258ae30db35c83fcc2f', 'request_id': 'req-d0aa3d4b-dbba-451a-935d-affa33603540'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:44.560277 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-d0aa3d4b-dbba-451a-935d-affa33603540 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ab311ac1d334258ae30db35c83fcc2f', 'request_id': 'req-d0aa3d4b-dbba-451a-935d-affa33603540'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:44.567703 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-d0aa3d4b-dbba-451a-935d-affa33603540 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ab311ac1d334258ae30db35c83fcc2f', 'request_id': 'req-d0aa3d4b-dbba-451a-935d-affa33603540'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:44.577935 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-d0aa3d4b-dbba-451a-935d-affa33603540 admin admin] The token cache is being invalidate because project 2ab311ac1d334258ae30db35c83fcc2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:01:44.578245 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-d0aa3d4b-dbba-451a-935d-affa33603540 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:44.578872 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-d0aa3d4b-dbba-451a-935d-affa33603540 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:44.580459 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1356/2737] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 13:01:44 2022] DELETE /identity/v3/projects/2ab311ac1d334258ae30db35c83fcc2f => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:01:44.585442 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17b6f8-d179-4dc3-927d-f1c4fc463842 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:44.585733 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17b6f8-d179-4dc3-927d-f1c4fc463842 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:44.586018 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17b6f8-d179-4dc3-927d-f1c4fc463842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:44.605928 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-0c17b6f8-d179-4dc3-927d-f1c4fc463842 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:01:44.607139 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-0c17b6f8-d179-4dc3-927d-f1c4fc463842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:44.669217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1382/2738] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 13:01:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:01:44.677184 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24be9ce-8992-43cc-8bec-ff35817cb2bb None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:44.677472 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24be9ce-8992-43cc-8bec-ff35817cb2bb None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:44.677751 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24be9ce-8992-43cc-8bec-ff35817cb2bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:44.697025 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-c24be9ce-8992-43cc-8bec-ff35817cb2bb None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:01:44.698224 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-c24be9ce-8992-43cc-8bec-ff35817cb2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:44.707231 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1357/2739] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 13:01:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 13:01:44.711329 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17b6f8-d179-4dc3-927d-f1c4fc463842 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:44.721696 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17b6f8-d179-4dc3-927d-f1c4fc463842 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:44.726124 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d488066a-8953-4f8e-8fd3-63c195fe9608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:44.726716 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d488066a-8953-4f8e-8fd3-63c195fe9608 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:44.727017 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d488066a-8953-4f8e-8fd3-63c195fe9608 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:44.727309 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d488066a-8953-4f8e-8fd3-63c195fe9608 admin admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:44.728029 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d488066a-8953-4f8e-8fd3-63c195fe9608 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:01:44.728335 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d488066a-8953-4f8e-8fd3-63c195fe9608 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:44.729917 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d488066a-8953-4f8e-8fd3-63c195fe9608 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:44.742961 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1383/2740] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 13:01:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:01:44.747410 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24be9ce-8992-43cc-8bec-ff35817cb2bb None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:44.756974 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24be9ce-8992-43cc-8bec-ff35817cb2bb None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:44.761955 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c6824a-0044-4322-a7d9-5fecf75be7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:44.762675 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6824a-0044-4322-a7d9-5fecf75be7e9 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:44.762964 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6824a-0044-4322-a7d9-5fecf75be7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:44.763241 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6824a-0044-4322-a7d9-5fecf75be7e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:44.764530 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c6824a-0044-4322-a7d9-5fecf75be7e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:44.766480 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c6824a-0044-4322-a7d9-5fecf75be7e9 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:44.789400 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:01:44.798422 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1358/2741] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 13:01:44 2022] POST /identity/v3/projects => generated 401 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:01:44.802552 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d488066a-8953-4f8e-8fd3-63c195fe9608 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:44.812387 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d488066a-8953-4f8e-8fd3-63c195fe9608 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:44.833369 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:01:44.869704 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de2b4c-46ff-4bdf-8fcb-f165861a025b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:44.870313 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de2b4c-46ff-4bdf-8fcb-f165861a025b admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:44.870590 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de2b4c-46ff-4bdf-8fcb-f165861a025b admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:44.870860 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de2b4c-46ff-4bdf-8fcb-f165861a025b admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:44.871520 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0de2b4c-46ff-4bdf-8fcb-f165861a025b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:44.873502 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0de2b4c-46ff-4bdf-8fcb-f165861a025b admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:44.895241 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1384/2742] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 13:01:44 2022] POST /identity/v3/users => generated 301 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:01:44.899395 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c6824a-0044-4322-a7d9-5fecf75be7e9 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:44.910608 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c6824a-0044-4322-a7d9-5fecf75be7e9 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:44.915331 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece4d7f9-dbf3-4489-9ddd-90d30c7feb4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:44.915987 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4d7f9-dbf3-4489-9ddd-90d30c7feb4b admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:44.916272 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4d7f9-dbf3-4489-9ddd-90d30c7feb4b admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:44.916550 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4d7f9-dbf3-4489-9ddd-90d30c7feb4b admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:44.917249 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece4d7f9-dbf3-4489-9ddd-90d30c7feb4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:44.918889 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece4d7f9-dbf3-4489-9ddd-90d30c7feb4b admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:44.929050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1359/2743] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 13:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:01:44.933080 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de2b4c-46ff-4bdf-8fcb-f165861a025b admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:44.943481 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de2b4c-46ff-4bdf-8fcb-f165861a025b admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:44.948063 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f9c95-283e-4a7d-a967-655365ac8e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:44.948687 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f9c95-283e-4a7d-a967-655365ac8e03 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:44.948964 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f9c95-283e-4a7d-a967-655365ac8e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:44.949232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f9c95-283e-4a7d-a967-655365ac8e03 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:44.949951 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927f9c95-283e-4a7d-a967-655365ac8e03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:44.952038 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927f9c95-283e-4a7d-a967-655365ac8e03 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:44.963586 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1385/2744] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 13:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:01:44.968319 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece4d7f9-dbf3-4489-9ddd-90d30c7feb4b admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:44.978376 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece4d7f9-dbf3-4489-9ddd-90d30c7feb4b admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:44.983173 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f102a9b3-0b2c-4b32-8046-740c635f4540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:44.983862 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102a9b3-0b2c-4b32-8046-740c635f4540 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:44.984148 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102a9b3-0b2c-4b32-8046-740c635f4540 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:44.984427 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102a9b3-0b2c-4b32-8046-740c635f4540 admin admin] PATH_INFO: `/v3/projects/55e36ac87cb84f09acc218e36bc2045e/users/ee0638ff8d674dcb8be3321efe90bcab/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:44.999438 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f102a9b3-0b2c-4b32-8046-740c635f4540 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55e36ac87cb84f09acc218e36bc2045e, user_id=ee0638ff8d674dcb8be3321efe90bcab, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:45.002153 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f102a9b3-0b2c-4b32-8046-740c635f4540 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:45.015891 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1360/2745] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 13:01:44 2022] PUT /identity/v3/projects/55e36ac87cb84f09acc218e36bc2045e/users/ee0638ff8d674dcb8be3321efe90bcab/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:01:45.027317 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647e8dc-01a4-41e6-9a0a-8663b379a87b None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:45.027602 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647e8dc-01a4-41e6-9a0a-8663b379a87b None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:45.027910 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647e8dc-01a4-41e6-9a0a-8663b379a87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:45.055055 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-e647e8dc-01a4-41e6-9a0a-8663b379a87b None None] MFA Rules not processed for user `ee0638ff8d674dcb8be3321efe90bcab`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:01:45.056306 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-e647e8dc-01a4-41e6-9a0a-8663b379a87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:45.120404 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1386/2746] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 13:01:45 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:01:45.125358 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f102a9b3-0b2c-4b32-8046-740c635f4540 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:45.191221 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f102a9b3-0b2c-4b32-8046-740c635f4540 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:45.196239 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823d01b3-ddba-45cd-b81f-cb2ec5987067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:45.196853 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823d01b3-ddba-45cd-b81f-cb2ec5987067 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:45.197136 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823d01b3-ddba-45cd-b81f-cb2ec5987067 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:45.197412 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823d01b3-ddba-45cd-b81f-cb2ec5987067 admin admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:45.198112 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823d01b3-ddba-45cd-b81f-cb2ec5987067 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:45.200049 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823d01b3-ddba-45cd-b81f-cb2ec5987067 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:45.227144 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1361/2747] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 13:01:45 2022] POST /identity/v3/projects => generated 403 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:01:45.231331 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e647e8dc-01a4-41e6-9a0a-8663b379a87b None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:45.241960 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e647e8dc-01a4-41e6-9a0a-8663b379a87b None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:45.297571 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f1bab-d9b7-40ed-affe-e67e2ff87acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:45.298179 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f1bab-d9b7-40ed-affe-e67e2ff87acf admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:45.298459 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f1bab-d9b7-40ed-affe-e67e2ff87acf admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:45.298729 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f1bab-d9b7-40ed-affe-e67e2ff87acf admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:45.299380 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f1bab-d9b7-40ed-affe-e67e2ff87acf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:45.301321 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f1bab-d9b7-40ed-affe-e67e2ff87acf admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:45.322345 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1387/2748] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 13:01:45 2022] POST /identity/v3/users => generated 308 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 13:01:45.327244 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823d01b3-ddba-45cd-b81f-cb2ec5987067 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:45.337722 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823d01b3-ddba-45cd-b81f-cb2ec5987067 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:45.342475 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af3e86-e8f7-48af-9966-0db81f6730b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:45.343069 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af3e86-e8f7-48af-9966-0db81f6730b5 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:45.343348 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af3e86-e8f7-48af-9966-0db81f6730b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:45.343626 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af3e86-e8f7-48af-9966-0db81f6730b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:45.344341 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af3e86-e8f7-48af-9966-0db81f6730b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:45.345915 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af3e86-e8f7-48af-9966-0db81f6730b5 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:45.356352 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1362/2749] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 13:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:01:45.361032 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f1bab-d9b7-40ed-affe-e67e2ff87acf admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:45.371017 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f1bab-d9b7-40ed-affe-e67e2ff87acf admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:45.376381 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a9efd-cf81-46bf-a8de-a251768d4674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:45.377021 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a9efd-cf81-46bf-a8de-a251768d4674 admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:45.377309 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a9efd-cf81-46bf-a8de-a251768d4674 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:45.377586 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a9efd-cf81-46bf-a8de-a251768d4674 admin admin] PATH_INFO: `/v3/projects/42caa060d0924b31af209b8d31363531/users/b528337415374c22aa9d31287727f5db/roles/0e78444e43c744668178971af4087ca0` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:45.392383 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2a9efd-cf81-46bf-a8de-a251768d4674 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42caa060d0924b31af209b8d31363531, user_id=b528337415374c22aa9d31287727f5db, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:45.395058 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2a9efd-cf81-46bf-a8de-a251768d4674 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:45.403128 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:01:45.409784 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1388/2750] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 13:01:45 2022] PUT /identity/v3/projects/42caa060d0924b31af209b8d31363531/users/b528337415374c22aa9d31287727f5db/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:01:45.413747 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af3e86-e8f7-48af-9966-0db81f6730b5 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:45.424226 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af3e86-e8f7-48af-9966-0db81f6730b5 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:45.479163 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e8d76c-1a57-491f-ba47-9ee13b0ba06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:45.479820 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8d76c-1a57-491f-ba47-9ee13b0ba06a admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:45.480149 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8d76c-1a57-491f-ba47-9ee13b0ba06a admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:45.480373 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8d76c-1a57-491f-ba47-9ee13b0ba06a admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:45.481065 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e8d76c-1a57-491f-ba47-9ee13b0ba06a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:45.483215 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e8d76c-1a57-491f-ba47-9ee13b0ba06a admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:45.493521 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1363/2751] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 13:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:01:45.497811 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a9efd-cf81-46bf-a8de-a251768d4674 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:45.508750 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a9efd-cf81-46bf-a8de-a251768d4674 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:45.513758 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a53b454-cd07-4b52-8c5c-492f5c559bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:45.514413 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a53b454-cd07-4b52-8c5c-492f5c559bfa admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:45.514770 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a53b454-cd07-4b52-8c5c-492f5c559bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:45.514974 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a53b454-cd07-4b52-8c5c-492f5c559bfa admin admin] PATH_INFO: `/v3/projects/42caa060d0924b31af209b8d31363531/users/b528337415374c22aa9d31287727f5db/roles/0e78444e43c744668178971af4087ca0` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:45.516940 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a53b454-cd07-4b52-8c5c-492f5c559bfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=42caa060d0924b31af209b8d31363531, user_id=b528337415374c22aa9d31287727f5db, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:45.519592 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a53b454-cd07-4b52-8c5c-492f5c559bfa admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:45.529847 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1389/2752] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 13:01:45 2022] PUT /identity/v3/projects/42caa060d0924b31af209b8d31363531/users/b528337415374c22aa9d31287727f5db/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:01:45.540032 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd3b7f-2827-44e1-9b7a-188eda2ab159 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:45.540322 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd3b7f-2827-44e1-9b7a-188eda2ab159 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:45.540601 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd3b7f-2827-44e1-9b7a-188eda2ab159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:45.568994 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-ecbd3b7f-2827-44e1-9b7a-188eda2ab159 None None] MFA Rules not processed for user `b528337415374c22aa9d31287727f5db`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:01:45.570198 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-ecbd3b7f-2827-44e1-9b7a-188eda2ab159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:45.589997 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:01:45.640130 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1364/2753] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 13:01:45 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:01:45.650893 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a53b454-cd07-4b52-8c5c-492f5c559bfa admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:45.652626 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-1a53b454-cd07-4b52-8c5c-492f5c559bfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:45.709466 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a53b454-cd07-4b52-8c5c-492f5c559bfa admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:45.713132 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651391b5-5b64-4320-8361-e5f9af6d68e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:45.713742 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651391b5-5b64-4320-8361-e5f9af6d68e3 service cinder] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:45.714024 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651391b5-5b64-4320-8361-e5f9af6d68e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:45.714300 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651391b5-5b64-4320-8361-e5f9af6d68e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:45.725780 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651391b5-5b64-4320-8361-e5f9af6d68e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:45.727276 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651391b5-5b64-4320-8361-e5f9af6d68e3 service cinder] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:45.737711 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1390/2754] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 13:01:45 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:01:45.871687 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd3b7f-2827-44e1-9b7a-188eda2ab159 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:45.934060 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd3b7f-2827-44e1-9b7a-188eda2ab159 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:45.939077 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a9046-3428-49cd-bd09-3fd0ca2253f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:45.939831 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a9046-3428-49cd-bd09-3fd0ca2253f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:45.940120 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a9046-3428-49cd-bd09-3fd0ca2253f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:45.940401 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a9046-3428-49cd-bd09-3fd0ca2253f4 admin admin] PATH_INFO: `/v3/users/ee0638ff8d674dcb8be3321efe90bcab` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:45.941566 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a9046-3428-49cd-bd09-3fd0ca2253f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee0638ff8d674dcb8be3321efe90bcab)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:45.943604 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a9046-3428-49cd-bd09-3fd0ca2253f4 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:45.990707 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-714a9046-3428-49cd-bd09-3fd0ca2253f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee0638ff8d674dcb8be3321efe90bcab'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:45.991299 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-714a9046-3428-49cd-bd09-3fd0ca2253f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee0638ff8d674dcb8be3321efe90bcab'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:45.991911 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-714a9046-3428-49cd-bd09-3fd0ca2253f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee0638ff8d674dcb8be3321efe90bcab'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:46.004727 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-714a9046-3428-49cd-bd09-3fd0ca2253f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee0638ff8d674dcb8be3321efe90bcab'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:46.011737 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-714a9046-3428-49cd-bd09-3fd0ca2253f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee0638ff8d674dcb8be3321efe90bcab'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:46.024250 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1365/2755] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 13:01:45 2022] DELETE /identity/v3/users/ee0638ff8d674dcb8be3321efe90bcab => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:01:46.028750 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747261ba-295b-405f-b084-cc367294d940 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:46.029760 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747261ba-295b-405f-b084-cc367294d940 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:46.029760 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747261ba-295b-405f-b084-cc367294d940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:46.049006 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-747261ba-295b-405f-b084-cc367294d940 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:01:46.050221 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-747261ba-295b-405f-b084-cc367294d940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:46.108992 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1391/2756] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 13:01:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:01:46.118783 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a9046-3428-49cd-bd09-3fd0ca2253f4 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:46.120247 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-714a9046-3428-49cd-bd09-3fd0ca2253f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:46.163687 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:01:46.180662 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a9046-3428-49cd-bd09-3fd0ca2253f4 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:46.184410 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1586de-812f-495e-ac7e-11cdc05b829e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:46.185023 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1586de-812f-495e-ac7e-11cdc05b829e service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:46.185307 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1586de-812f-495e-ac7e-11cdc05b829e service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:46.185611 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1586de-812f-495e-ac7e-11cdc05b829e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:46.194822 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1586de-812f-495e-ac7e-11cdc05b829e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:46.196286 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1586de-812f-495e-ac7e-11cdc05b829e service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:46.206192 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1366/2757] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 13:01:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:01:46.473097 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747261ba-295b-405f-b084-cc367294d940 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:46.474674 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-747261ba-295b-405f-b084-cc367294d940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:46.486645 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747261ba-295b-405f-b084-cc367294d940 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:46.491193 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c9cfc2-92c6-4cf5-86b7-d9548d83633f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:46.491857 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9cfc2-92c6-4cf5-86b7-d9548d83633f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:46.492135 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9cfc2-92c6-4cf5-86b7-d9548d83633f admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:46.492405 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9cfc2-92c6-4cf5-86b7-d9548d83633f admin admin] PATH_INFO: `/v3/projects/55e36ac87cb84f09acc218e36bc2045e` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:46.493573 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c9cfc2-92c6-4cf5-86b7-d9548d83633f admin admin] RBAC: Authorizing `identity:delete_project(project_id=55e36ac87cb84f09acc218e36bc2045e)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:46.495480 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c9cfc2-92c6-4cf5-86b7-d9548d83633f admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:46.551356 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-43c9cfc2-92c6-4cf5-86b7-d9548d83633f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55e36ac87cb84f09acc218e36bc2045e', 'request_id': 'req-43c9cfc2-92c6-4cf5-86b7-d9548d83633f'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:46.557579 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-43c9cfc2-92c6-4cf5-86b7-d9548d83633f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55e36ac87cb84f09acc218e36bc2045e', 'request_id': 'req-43c9cfc2-92c6-4cf5-86b7-d9548d83633f'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:46.565990 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-43c9cfc2-92c6-4cf5-86b7-d9548d83633f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55e36ac87cb84f09acc218e36bc2045e', 'request_id': 'req-43c9cfc2-92c6-4cf5-86b7-d9548d83633f'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:46.577048 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-43c9cfc2-92c6-4cf5-86b7-d9548d83633f admin admin] The token cache is being invalidate because project 55e36ac87cb84f09acc218e36bc2045e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:01:46.577335 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-43c9cfc2-92c6-4cf5-86b7-d9548d83633f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:46.578090 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-43c9cfc2-92c6-4cf5-86b7-d9548d83633f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:46.579606 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1392/2758] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 13:01:46 2022] DELETE /identity/v3/projects/55e36ac87cb84f09acc218e36bc2045e => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:01:46.583526 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1586de-812f-495e-ac7e-11cdc05b829e service neutron] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:46.593476 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1586de-812f-495e-ac7e-11cdc05b829e service neutron] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:46.652468 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d89dc7-02dd-4203-b586-97826b106fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:46.653099 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d89dc7-02dd-4203-b586-97826b106fc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:46.653382 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d89dc7-02dd-4203-b586-97826b106fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:46.653658 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d89dc7-02dd-4203-b586-97826b106fc5 admin admin] PATH_INFO: `/v3/users/b528337415374c22aa9d31287727f5db` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:46.654905 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d89dc7-02dd-4203-b586-97826b106fc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b528337415374c22aa9d31287727f5db)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:46.656916 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d89dc7-02dd-4203-b586-97826b106fc5 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:46.702695 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-56d89dc7-02dd-4203-b586-97826b106fc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b528337415374c22aa9d31287727f5db'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:46.703334 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-56d89dc7-02dd-4203-b586-97826b106fc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b528337415374c22aa9d31287727f5db'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:46.703952 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-56d89dc7-02dd-4203-b586-97826b106fc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b528337415374c22aa9d31287727f5db'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:46.715619 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-56d89dc7-02dd-4203-b586-97826b106fc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b528337415374c22aa9d31287727f5db'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:46.722325 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-56d89dc7-02dd-4203-b586-97826b106fc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b528337415374c22aa9d31287727f5db'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:46.735180 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1367/2759] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 13:01:46 2022] DELETE /identity/v3/users/b528337415374c22aa9d31287727f5db => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:01:47.006773 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c9cfc2-92c6-4cf5-86b7-d9548d83633f admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:47.008688 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-43c9cfc2-92c6-4cf5-86b7-d9548d83633f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:47.027084 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c9cfc2-92c6-4cf5-86b7-d9548d83633f admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:47.035294 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:01:47.092105 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc574a16-e60b-4e76-b55e-525ce7821ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:47.092709 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc574a16-e60b-4e76-b55e-525ce7821ec6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:47.092990 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc574a16-e60b-4e76-b55e-525ce7821ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:47.093270 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc574a16-e60b-4e76-b55e-525ce7821ec6 admin admin] PATH_INFO: `/v3/projects/42caa060d0924b31af209b8d31363531` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:47.094448 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc574a16-e60b-4e76-b55e-525ce7821ec6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42caa060d0924b31af209b8d31363531)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:47.096444 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc574a16-e60b-4e76-b55e-525ce7821ec6 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:47.143465 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-cc574a16-e60b-4e76-b55e-525ce7821ec6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42caa060d0924b31af209b8d31363531', 'request_id': 'req-cc574a16-e60b-4e76-b55e-525ce7821ec6'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:47.149651 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-cc574a16-e60b-4e76-b55e-525ce7821ec6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42caa060d0924b31af209b8d31363531', 'request_id': 'req-cc574a16-e60b-4e76-b55e-525ce7821ec6'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:47.156916 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-cc574a16-e60b-4e76-b55e-525ce7821ec6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42caa060d0924b31af209b8d31363531', 'request_id': 'req-cc574a16-e60b-4e76-b55e-525ce7821ec6'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:47.171352 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-cc574a16-e60b-4e76-b55e-525ce7821ec6 admin admin] The token cache is being invalidate because project 42caa060d0924b31af209b8d31363531 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:01:47.171664 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-cc574a16-e60b-4e76-b55e-525ce7821ec6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:47.172317 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-cc574a16-e60b-4e76-b55e-525ce7821ec6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:47.173847 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1393/2760] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 13:01:47 2022] DELETE /identity/v3/projects/42caa060d0924b31af209b8d31363531 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:01:47.178842 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6c0c1-ca4d-4a98-883e-9b8608b82631 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:47.179345 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6c0c1-ca4d-4a98-883e-9b8608b82631 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:47.179461 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6c0c1-ca4d-4a98-883e-9b8608b82631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:47.200892 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-31e6c0c1-ca4d-4a98-883e-9b8608b82631 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:01:47.202121 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-31e6c0c1-ca4d-4a98-883e-9b8608b82631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:47.252799 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:01:47.263100 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1368/2761] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 13:01:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:01:47.271926 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa552439-50cc-4ce3-9239-96d394b9fd40 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:47.272218 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa552439-50cc-4ce3-9239-96d394b9fd40 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:47.272494 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa552439-50cc-4ce3-9239-96d394b9fd40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:47.293432 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-aa552439-50cc-4ce3-9239-96d394b9fd40 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:01:47.294648 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-aa552439-50cc-4ce3-9239-96d394b9fd40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:47.303529 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1394/2762] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 13:01:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:01:47.307707 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e6c0c1-ca4d-4a98-883e-9b8608b82631 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:47.317456 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e6c0c1-ca4d-4a98-883e-9b8608b82631 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:47.321979 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea4b375-5d0b-49a9-addc-d8379b1f428f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:47.322578 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea4b375-5d0b-49a9-addc-d8379b1f428f admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:47.322852 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea4b375-5d0b-49a9-addc-d8379b1f428f admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:47.323120 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea4b375-5d0b-49a9-addc-d8379b1f428f admin admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:47.323840 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea4b375-5d0b-49a9-addc-d8379b1f428f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:01:47.324144 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea4b375-5d0b-49a9-addc-d8379b1f428f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:47.325756 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea4b375-5d0b-49a9-addc-d8379b1f428f admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:47.337142 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1369/2763] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 13:01:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:01:47.341328 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa552439-50cc-4ce3-9239-96d394b9fd40 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:47.351259 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa552439-50cc-4ce3-9239-96d394b9fd40 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:47.355932 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff622a1-e6ca-429a-9c09-768e76cf99f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:47.356533 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff622a1-e6ca-429a-9c09-768e76cf99f2 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:47.356814 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff622a1-e6ca-429a-9c09-768e76cf99f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:47.357088 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff622a1-e6ca-429a-9c09-768e76cf99f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:47.358345 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff622a1-e6ca-429a-9c09-768e76cf99f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:47.360310 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff622a1-e6ca-429a-9c09-768e76cf99f2 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:47.390042 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1395/2764] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 13:01:47 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:01:47.394224 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea4b375-5d0b-49a9-addc-d8379b1f428f admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:47.404667 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea4b375-5d0b-49a9-addc-d8379b1f428f admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:47.460932 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b11568-e557-4f60-85dc-1808af569f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:47.461542 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b11568-e557-4f60-85dc-1808af569f65 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:47.461825 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b11568-e557-4f60-85dc-1808af569f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:47.462099 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b11568-e557-4f60-85dc-1808af569f65 admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:47.462756 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b11568-e557-4f60-85dc-1808af569f65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:47.464722 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b11568-e557-4f60-85dc-1808af569f65 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:47.488112 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1370/2765] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 13:01:47 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:01:47.492221 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff622a1-e6ca-429a-9c09-768e76cf99f2 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:47.502776 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff622a1-e6ca-429a-9c09-768e76cf99f2 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:47.507377 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0908e5c-4b0f-41c1-adb0-fdfac1bb16d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:47.508013 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0908e5c-4b0f-41c1-adb0-fdfac1bb16d3 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:47.508298 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0908e5c-4b0f-41c1-adb0-fdfac1bb16d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:47.508574 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0908e5c-4b0f-41c1-adb0-fdfac1bb16d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:47.509268 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0908e5c-4b0f-41c1-adb0-fdfac1bb16d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:47.511259 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0908e5c-4b0f-41c1-adb0-fdfac1bb16d3 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:47.522064 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1396/2766] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 13:01:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:01:47.526313 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b11568-e557-4f60-85dc-1808af569f65 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:47.537014 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b11568-e557-4f60-85dc-1808af569f65 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:47.541723 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff2167-9f25-428a-bdde-b9c2a0d5274f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:47.542342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff2167-9f25-428a-bdde-b9c2a0d5274f admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:47.542622 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff2167-9f25-428a-bdde-b9c2a0d5274f admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:47.542890 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff2167-9f25-428a-bdde-b9c2a0d5274f admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:47.543596 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dff2167-9f25-428a-bdde-b9c2a0d5274f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:47.545831 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dff2167-9f25-428a-bdde-b9c2a0d5274f admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:47.556667 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1371/2767] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 13:01:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:01:47.560785 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0908e5c-4b0f-41c1-adb0-fdfac1bb16d3 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:47.571042 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0908e5c-4b0f-41c1-adb0-fdfac1bb16d3 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:47.575562 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f1a089-0243-4759-8fc7-f49fe295b6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:47.576242 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1a089-0243-4759-8fc7-f49fe295b6c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:47.576524 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1a089-0243-4759-8fc7-f49fe295b6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:47.576796 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1a089-0243-4759-8fc7-f49fe295b6c7 admin admin] PATH_INFO: `/v3/projects/bb1fb7afcff64519b9ca4089e50afed0/users/69aa30c27263443aa29b32fc6b60c475/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:47.591081 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f1a089-0243-4759-8fc7-f49fe295b6c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb1fb7afcff64519b9ca4089e50afed0, user_id=69aa30c27263443aa29b32fc6b60c475, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:47.594502 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f1a089-0243-4759-8fc7-f49fe295b6c7 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:47.608603 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1397/2768] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 13:01:47 2022] PUT /identity/v3/projects/bb1fb7afcff64519b9ca4089e50afed0/users/69aa30c27263443aa29b32fc6b60c475/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:01:47.672792 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f79333-c58f-4271-9398-a8931787a012 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:47.673080 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f79333-c58f-4271-9398-a8931787a012 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:47.673359 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f79333-c58f-4271-9398-a8931787a012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:47.700897 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-17f79333-c58f-4271-9398-a8931787a012 None None] MFA Rules not processed for user `69aa30c27263443aa29b32fc6b60c475`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:01:47.702586 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-17f79333-c58f-4271-9398-a8931787a012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:47.770847 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1372/2769] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 13:01:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:01:47.775547 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f1a089-0243-4759-8fc7-f49fe295b6c7 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:47.818482 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:01:47.841353 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f1a089-0243-4759-8fc7-f49fe295b6c7 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:47.846708 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211bbd4d-8c24-494c-9a78-14b1b90bf074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:47.847320 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211bbd4d-8c24-494c-9a78-14b1b90bf074 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:47.847600 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211bbd4d-8c24-494c-9a78-14b1b90bf074 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:47.847910 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211bbd4d-8c24-494c-9a78-14b1b90bf074 admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:47.848574 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211bbd4d-8c24-494c-9a78-14b1b90bf074 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:47.850474 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211bbd4d-8c24-494c-9a78-14b1b90bf074 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:47.877967 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1398/2770] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 13:01:47 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:01:47.882141 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f79333-c58f-4271-9398-a8931787a012 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:47.893891 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f79333-c58f-4271-9398-a8931787a012 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:47.949287 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:01:47.958176 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30828310-82d4-4c56-a6a1-7fbeb13cb131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:47.958796 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30828310-82d4-4c56-a6a1-7fbeb13cb131 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:47.959080 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30828310-82d4-4c56-a6a1-7fbeb13cb131 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:47.959354 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30828310-82d4-4c56-a6a1-7fbeb13cb131 admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:47.960046 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30828310-82d4-4c56-a6a1-7fbeb13cb131 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:47.961991 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30828310-82d4-4c56-a6a1-7fbeb13cb131 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:47.982771 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1373/2771] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 13:01:47 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:01:47.986799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211bbd4d-8c24-494c-9a78-14b1b90bf074 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:47.997232 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211bbd4d-8c24-494c-9a78-14b1b90bf074 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:48.002322 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183aba1c-5a1c-4d7a-9207-71d1aa0a2610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:48.002922 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183aba1c-5a1c-4d7a-9207-71d1aa0a2610 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:48.003199 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183aba1c-5a1c-4d7a-9207-71d1aa0a2610 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:48.003472 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183aba1c-5a1c-4d7a-9207-71d1aa0a2610 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:48.004606 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183aba1c-5a1c-4d7a-9207-71d1aa0a2610 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:48.006202 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183aba1c-5a1c-4d7a-9207-71d1aa0a2610 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:48.017507 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1399/2772] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 13:01:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:01:48.021791 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30828310-82d4-4c56-a6a1-7fbeb13cb131 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:48.031741 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30828310-82d4-4c56-a6a1-7fbeb13cb131 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:48.036353 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a4eaa-9ae9-4c90-9a2b-6ad10ccbde19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:48.036998 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a4eaa-9ae9-4c90-9a2b-6ad10ccbde19 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:48.037279 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a4eaa-9ae9-4c90-9a2b-6ad10ccbde19 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:48.037555 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a4eaa-9ae9-4c90-9a2b-6ad10ccbde19 admin admin] PATH_INFO: `/v3/projects/6966512d541346b48510c299999f0b52/users/111835c2d218432c8c5c805302a2e418/roles/0e78444e43c744668178971af4087ca0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:48.051881 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a4eaa-9ae9-4c90-9a2b-6ad10ccbde19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6966512d541346b48510c299999f0b52, user_id=111835c2d218432c8c5c805302a2e418, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:48.054835 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a4eaa-9ae9-4c90-9a2b-6ad10ccbde19 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:48.068907 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1374/2773] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 13:01:48 2022] PUT /identity/v3/projects/6966512d541346b48510c299999f0b52/users/111835c2d218432c8c5c805302a2e418/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:01:48.072876 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183aba1c-5a1c-4d7a-9207-71d1aa0a2610 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:48.083431 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183aba1c-5a1c-4d7a-9207-71d1aa0a2610 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:48.138127 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3e5c2-d4dc-44fa-b542-93a229f878a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:48.139212 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3e5c2-d4dc-44fa-b542-93a229f878a7 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:48.139473 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3e5c2-d4dc-44fa-b542-93a229f878a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:48.140197 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3e5c2-d4dc-44fa-b542-93a229f878a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:48.140887 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3e5c2-d4dc-44fa-b542-93a229f878a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:48.142471 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3e5c2-d4dc-44fa-b542-93a229f878a7 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:48.152586 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1400/2774] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 13:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:01:48.156824 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a4eaa-9ae9-4c90-9a2b-6ad10ccbde19 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:48.172305 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a4eaa-9ae9-4c90-9a2b-6ad10ccbde19 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:48.177510 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe4ccb-9e98-417a-ae95-2514c85cbde7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:48.178184 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe4ccb-9e98-417a-ae95-2514c85cbde7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:48.178464 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe4ccb-9e98-417a-ae95-2514c85cbde7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:48.178737 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe4ccb-9e98-417a-ae95-2514c85cbde7 admin admin] PATH_INFO: `/v3/projects/6966512d541346b48510c299999f0b52/users/111835c2d218432c8c5c805302a2e418/roles/0e78444e43c744668178971af4087ca0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:48.180943 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfe4ccb-9e98-417a-ae95-2514c85cbde7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6966512d541346b48510c299999f0b52, user_id=111835c2d218432c8c5c805302a2e418, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:48.183808 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfe4ccb-9e98-417a-ae95-2514c85cbde7 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:48.195129 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1375/2775] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 13:01:48 2022] PUT /identity/v3/projects/6966512d541346b48510c299999f0b52/users/111835c2d218432c8c5c805302a2e418/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:01:48.206094 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7af11-4758-4ca0-a908-9e3d40907bbc None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:48.206380 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7af11-4758-4ca0-a908-9e3d40907bbc None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:48.206671 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7af11-4758-4ca0-a908-9e3d40907bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:48.233707 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-e6d7af11-4758-4ca0-a908-9e3d40907bbc None None] MFA Rules not processed for user `111835c2d218432c8c5c805302a2e418`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:01:48.234900 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-e6d7af11-4758-4ca0-a908-9e3d40907bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:48.302053 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1401/2776] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 13:01:48 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:01:48.313915 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe4ccb-9e98-417a-ae95-2514c85cbde7 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:48.315598 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-ebfe4ccb-9e98-417a-ae95-2514c85cbde7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:48.375339 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe4ccb-9e98-417a-ae95-2514c85cbde7 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:48.379143 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0980c3b-ec5b-4f6c-a116-b893c08d81a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:48.379788 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0980c3b-ec5b-4f6c-a116-b893c08d81a7 service cinder] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:48.380072 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0980c3b-ec5b-4f6c-a116-b893c08d81a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:48.380348 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0980c3b-ec5b-4f6c-a116-b893c08d81a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:48.389721 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0980c3b-ec5b-4f6c-a116-b893c08d81a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:48.391095 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0980c3b-ec5b-4f6c-a116-b893c08d81a7 service cinder] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:48.400663 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1376/2777] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 13:01:48 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:01:48.788241 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7af11-4758-4ca0-a908-9e3d40907bbc None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:48.821847 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:01:48.851790 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7af11-4758-4ca0-a908-9e3d40907bbc None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:48.856354 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73894dbf-cfa6-47b1-8405-14fffe7d2a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:48.856988 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73894dbf-cfa6-47b1-8405-14fffe7d2a21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:48.857308 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73894dbf-cfa6-47b1-8405-14fffe7d2a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:48.858077 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73894dbf-cfa6-47b1-8405-14fffe7d2a21 admin admin] PATH_INFO: `/v3/users/69aa30c27263443aa29b32fc6b60c475` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:48.859251 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73894dbf-cfa6-47b1-8405-14fffe7d2a21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69aa30c27263443aa29b32fc6b60c475)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:48.861222 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73894dbf-cfa6-47b1-8405-14fffe7d2a21 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:48.912183 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-73894dbf-cfa6-47b1-8405-14fffe7d2a21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69aa30c27263443aa29b32fc6b60c475'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:48.919620 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-73894dbf-cfa6-47b1-8405-14fffe7d2a21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69aa30c27263443aa29b32fc6b60c475'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:48.920837 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-73894dbf-cfa6-47b1-8405-14fffe7d2a21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69aa30c27263443aa29b32fc6b60c475'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:48.927896 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-73894dbf-cfa6-47b1-8405-14fffe7d2a21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69aa30c27263443aa29b32fc6b60c475'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:48.941285 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-73894dbf-cfa6-47b1-8405-14fffe7d2a21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69aa30c27263443aa29b32fc6b60c475'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:48.948202 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1402/2778] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 13:01:48 2022] DELETE /identity/v3/users/69aa30c27263443aa29b32fc6b60c475 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:01:48.952604 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40872f48-100c-4a3d-a4a5-93c502284134 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:48.952931 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40872f48-100c-4a3d-a4a5-93c502284134 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:48.953206 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40872f48-100c-4a3d-a4a5-93c502284134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:48.972782 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-40872f48-100c-4a3d-a4a5-93c502284134 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:01:48.973984 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-40872f48-100c-4a3d-a4a5-93c502284134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:49.032196 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1377/2779] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 13:01:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:01:49.041813 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73894dbf-cfa6-47b1-8405-14fffe7d2a21 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:49.043230 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-73894dbf-cfa6-47b1-8405-14fffe7d2a21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:49.102756 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73894dbf-cfa6-47b1-8405-14fffe7d2a21 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:49.106484 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4a1a5-85b7-4f37-b30d-5843de95ded7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:49.107089 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4a1a5-85b7-4f37-b30d-5843de95ded7 service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:49.108089 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4a1a5-85b7-4f37-b30d-5843de95ded7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:49.108367 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4a1a5-85b7-4f37-b30d-5843de95ded7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:49.117386 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe4a1a5-85b7-4f37-b30d-5843de95ded7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:49.118818 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe4a1a5-85b7-4f37-b30d-5843de95ded7 service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:49.128519 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1403/2780] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 13:01:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:01:49.405860 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40872f48-100c-4a3d-a4a5-93c502284134 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:49.407356 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-40872f48-100c-4a3d-a4a5-93c502284134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:49.422668 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40872f48-100c-4a3d-a4a5-93c502284134 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:49.430189 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b540c40-12f4-4e49-b4a0-76c5bda73efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:49.430731 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b540c40-12f4-4e49-b4a0-76c5bda73efe admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:49.431014 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b540c40-12f4-4e49-b4a0-76c5bda73efe admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:49.431561 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b540c40-12f4-4e49-b4a0-76c5bda73efe admin admin] PATH_INFO: `/v3/projects/bb1fb7afcff64519b9ca4089e50afed0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:49.432769 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b540c40-12f4-4e49-b4a0-76c5bda73efe admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb1fb7afcff64519b9ca4089e50afed0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:49.434726 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b540c40-12f4-4e49-b4a0-76c5bda73efe admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:49.481485 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:01:49.485172 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-3b540c40-12f4-4e49-b4a0-76c5bda73efe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb1fb7afcff64519b9ca4089e50afed0', 'request_id': 'req-3b540c40-12f4-4e49-b4a0-76c5bda73efe'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:49.492942 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-3b540c40-12f4-4e49-b4a0-76c5bda73efe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb1fb7afcff64519b9ca4089e50afed0', 'request_id': 'req-3b540c40-12f4-4e49-b4a0-76c5bda73efe'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:49.500774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-3b540c40-12f4-4e49-b4a0-76c5bda73efe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb1fb7afcff64519b9ca4089e50afed0', 'request_id': 'req-3b540c40-12f4-4e49-b4a0-76c5bda73efe'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:49.512008 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-3b540c40-12f4-4e49-b4a0-76c5bda73efe admin admin] The token cache is being invalidate because project bb1fb7afcff64519b9ca4089e50afed0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:01:49.512376 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-3b540c40-12f4-4e49-b4a0-76c5bda73efe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:49.513092 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-3b540c40-12f4-4e49-b4a0-76c5bda73efe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:49.515291 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1378/2781] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 13:01:49 2022] DELETE /identity/v3/projects/bb1fb7afcff64519b9ca4089e50afed0 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 13:01:49.519404 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4a1a5-85b7-4f37-b30d-5843de95ded7 service neutron] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:49.529093 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4a1a5-85b7-4f37-b30d-5843de95ded7 service neutron] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:49.587291 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:01:49.595548 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a314fa53-9ff6-48c3-bcbf-d2da9c00a542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:49.596632 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314fa53-9ff6-48c3-bcbf-d2da9c00a542 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:49.596985 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314fa53-9ff6-48c3-bcbf-d2da9c00a542 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:49.597299 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314fa53-9ff6-48c3-bcbf-d2da9c00a542 admin admin] PATH_INFO: `/v3/users/111835c2d218432c8c5c805302a2e418` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:49.598642 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a314fa53-9ff6-48c3-bcbf-d2da9c00a542 admin admin] RBAC: Authorizing `identity:delete_user(user_id=111835c2d218432c8c5c805302a2e418)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:49.600879 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a314fa53-9ff6-48c3-bcbf-d2da9c00a542 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:49.648408 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-a314fa53-9ff6-48c3-bcbf-d2da9c00a542 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '111835c2d218432c8c5c805302a2e418'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:49.657037 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-a314fa53-9ff6-48c3-bcbf-d2da9c00a542 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '111835c2d218432c8c5c805302a2e418'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:49.657640 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-a314fa53-9ff6-48c3-bcbf-d2da9c00a542 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '111835c2d218432c8c5c805302a2e418'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:49.664862 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-a314fa53-9ff6-48c3-bcbf-d2da9c00a542 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '111835c2d218432c8c5c805302a2e418'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:49.677087 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-a314fa53-9ff6-48c3-bcbf-d2da9c00a542 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '111835c2d218432c8c5c805302a2e418'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:49.683056 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1404/2782] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 13:01:49 2022] DELETE /identity/v3/users/111835c2d218432c8c5c805302a2e418 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:01:49.976732 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b540c40-12f4-4e49-b4a0-76c5bda73efe admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:49.978634 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-3b540c40-12f4-4e49-b4a0-76c5bda73efe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:50.000239 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b540c40-12f4-4e49-b4a0-76c5bda73efe admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:50.071851 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c35c12-9c10-43ed-8b66-3ff22e39853b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:50.072666 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c35c12-9c10-43ed-8b66-3ff22e39853b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:50.073118 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c35c12-9c10-43ed-8b66-3ff22e39853b admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:50.073556 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c35c12-9c10-43ed-8b66-3ff22e39853b admin admin] PATH_INFO: `/v3/projects/6966512d541346b48510c299999f0b52` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:50.075081 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c35c12-9c10-43ed-8b66-3ff22e39853b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6966512d541346b48510c299999f0b52)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:50.077831 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c35c12-9c10-43ed-8b66-3ff22e39853b admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:50.127170 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-b0c35c12-9c10-43ed-8b66-3ff22e39853b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6966512d541346b48510c299999f0b52', 'request_id': 'req-b0c35c12-9c10-43ed-8b66-3ff22e39853b'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:50.133696 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-b0c35c12-9c10-43ed-8b66-3ff22e39853b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6966512d541346b48510c299999f0b52', 'request_id': 'req-b0c35c12-9c10-43ed-8b66-3ff22e39853b'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:50.142327 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-b0c35c12-9c10-43ed-8b66-3ff22e39853b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6966512d541346b48510c299999f0b52', 'request_id': 'req-b0c35c12-9c10-43ed-8b66-3ff22e39853b'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:50.154607 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-b0c35c12-9c10-43ed-8b66-3ff22e39853b admin admin] The token cache is being invalidate because project 6966512d541346b48510c299999f0b52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:01:50.154914 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-b0c35c12-9c10-43ed-8b66-3ff22e39853b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:50.155549 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-b0c35c12-9c10-43ed-8b66-3ff22e39853b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:50.157123 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1379/2783] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 13:01:49 2022] DELETE /identity/v3/projects/6966512d541346b48510c299999f0b52 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:01:50.162291 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f965d11-e2eb-45db-99a9-66b159f20b52 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:50.163149 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f965d11-e2eb-45db-99a9-66b159f20b52 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:50.163149 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f965d11-e2eb-45db-99a9-66b159f20b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:50.186199 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-2f965d11-e2eb-45db-99a9-66b159f20b52 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:01:50.187420 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-2f965d11-e2eb-45db-99a9-66b159f20b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:50.245256 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1405/2784] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 13:01:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:01:50.255164 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f318b73-935a-4d42-ada5-757f5af493a8 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:50.255456 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f318b73-935a-4d42-ada5-757f5af493a8 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:50.255777 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f318b73-935a-4d42-ada5-757f5af493a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:50.275747 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-2f318b73-935a-4d42-ada5-757f5af493a8 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:01:50.278153 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-2f318b73-935a-4d42-ada5-757f5af493a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:50.285677 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1380/2785] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 13:01:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:01:50.290240 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f965d11-e2eb-45db-99a9-66b159f20b52 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:50.300894 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f965d11-e2eb-45db-99a9-66b159f20b52 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:50.305600 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092f5662-ab69-4a1d-b905-9991831e9270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:50.306209 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f5662-ab69-4a1d-b905-9991831e9270 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:50.306490 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f5662-ab69-4a1d-b905-9991831e9270 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:50.306764 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f5662-ab69-4a1d-b905-9991831e9270 admin admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:50.307488 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092f5662-ab69-4a1d-b905-9991831e9270 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:01:50.307821 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092f5662-ab69-4a1d-b905-9991831e9270 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:50.309427 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092f5662-ab69-4a1d-b905-9991831e9270 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:50.321042 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1406/2786] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 13:01:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:01:50.325198 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f318b73-935a-4d42-ada5-757f5af493a8 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:50.334736 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f318b73-935a-4d42-ada5-757f5af493a8 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:50.339345 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7299d7-136e-41bc-a868-0bd2e312fd29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:50.339966 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7299d7-136e-41bc-a868-0bd2e312fd29 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:50.340242 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7299d7-136e-41bc-a868-0bd2e312fd29 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:50.340518 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7299d7-136e-41bc-a868-0bd2e312fd29 admin admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:50.341187 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7299d7-136e-41bc-a868-0bd2e312fd29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:50.343567 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7299d7-136e-41bc-a868-0bd2e312fd29 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:50.371028 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1381/2787] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 13:01:50 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:01:50.374958 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092f5662-ab69-4a1d-b905-9991831e9270 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:50.385568 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092f5662-ab69-4a1d-b905-9991831e9270 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:50.443944 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d5b54-7a0c-4d39-b16a-daea685bf272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:50.444556 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d5b54-7a0c-4d39-b16a-daea685bf272 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:50.444841 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d5b54-7a0c-4d39-b16a-daea685bf272 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:50.445135 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d5b54-7a0c-4d39-b16a-daea685bf272 admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:50.446166 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1d5b54-7a0c-4d39-b16a-daea685bf272 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:50.447866 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1d5b54-7a0c-4d39-b16a-daea685bf272 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:50.456619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:01:50.470843 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1407/2788] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 13:01:50 2022] POST /identity/v3/users => generated 284 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 13:01:50.475139 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7299d7-136e-41bc-a868-0bd2e312fd29 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:50.486057 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7299d7-136e-41bc-a868-0bd2e312fd29 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:50.490704 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4720de-9e15-4721-9a16-37a88a2b54f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:50.491307 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4720de-9e15-4721-9a16-37a88a2b54f2 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:50.491590 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4720de-9e15-4721-9a16-37a88a2b54f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:50.491894 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4720de-9e15-4721-9a16-37a88a2b54f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:50.492591 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4720de-9e15-4721-9a16-37a88a2b54f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:50.494196 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4720de-9e15-4721-9a16-37a88a2b54f2 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:50.504141 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1382/2789] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 13:01:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:01:50.508168 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d5b54-7a0c-4d39-b16a-daea685bf272 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:50.518317 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d5b54-7a0c-4d39-b16a-daea685bf272 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:50.522854 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2019cbed-1174-4df5-920b-00d706b2a15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:50.523477 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019cbed-1174-4df5-920b-00d706b2a15c admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:50.523811 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019cbed-1174-4df5-920b-00d706b2a15c admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:50.524067 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019cbed-1174-4df5-920b-00d706b2a15c admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:50.524753 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2019cbed-1174-4df5-920b-00d706b2a15c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:50.526364 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2019cbed-1174-4df5-920b-00d706b2a15c admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:50.537314 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1408/2790] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 13:01:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:01:50.541514 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4720de-9e15-4721-9a16-37a88a2b54f2 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:50.552241 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4720de-9e15-4721-9a16-37a88a2b54f2 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:50.556967 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d26c57e-caff-465e-99c1-f4dc54b4f029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:50.557690 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d26c57e-caff-465e-99c1-f4dc54b4f029 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:50.557974 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d26c57e-caff-465e-99c1-f4dc54b4f029 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:50.558249 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d26c57e-caff-465e-99c1-f4dc54b4f029 admin admin] PATH_INFO: `/v3/projects/1cf43929a1b0493c861d4f76d9d29f79/users/53d22983e68b48bb9d3b87cde8b79904/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:50.573536 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d26c57e-caff-465e-99c1-f4dc54b4f029 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cf43929a1b0493c861d4f76d9d29f79, user_id=53d22983e68b48bb9d3b87cde8b79904, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:50.576309 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d26c57e-caff-465e-99c1-f4dc54b4f029 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:50.585113 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:01:50.590626 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1383/2791] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 13:01:50 2022] PUT /identity/v3/projects/1cf43929a1b0493c861d4f76d9d29f79/users/53d22983e68b48bb9d3b87cde8b79904/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:01:50.601700 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7cf76-e248-4398-ae57-397eb60dd364 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:50.601984 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7cf76-e248-4398-ae57-397eb60dd364 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:50.602255 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7cf76-e248-4398-ae57-397eb60dd364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:50.629893 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-11b7cf76-e248-4398-ae57-397eb60dd364 None None] MFA Rules not processed for user `53d22983e68b48bb9d3b87cde8b79904`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:01:50.631115 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-11b7cf76-e248-4398-ae57-397eb60dd364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:50.695440 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1409/2792] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 13:01:50 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:01:50.700730 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d26c57e-caff-465e-99c1-f4dc54b4f029 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:50.761470 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d26c57e-caff-465e-99c1-f4dc54b4f029 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:50.766757 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b7b02d-e859-4655-a228-b657f9cdfa80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:50.767362 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7b02d-e859-4655-a228-b657f9cdfa80 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:50.767679 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7b02d-e859-4655-a228-b657f9cdfa80 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:50.767951 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7b02d-e859-4655-a228-b657f9cdfa80 admin admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:50.768617 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b7b02d-e859-4655-a228-b657f9cdfa80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:50.770512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b7b02d-e859-4655-a228-b657f9cdfa80 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:50.797842 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1384/2793] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 13:01:50 2022] POST /identity/v3/projects => generated 365 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:01:50.802115 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b7cf76-e248-4398-ae57-397eb60dd364 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:50.812977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b7cf76-e248-4398-ae57-397eb60dd364 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:50.884693 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd22e735-8740-4ed0-a6d8-299f2c82ff30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:50.885313 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22e735-8740-4ed0-a6d8-299f2c82ff30 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:50.885594 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22e735-8740-4ed0-a6d8-299f2c82ff30 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:50.885988 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22e735-8740-4ed0-a6d8-299f2c82ff30 admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:50.886678 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd22e735-8740-4ed0-a6d8-299f2c82ff30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:50.889133 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd22e735-8740-4ed0-a6d8-299f2c82ff30 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:50.914147 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1410/2794] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 13:01:50 2022] POST /identity/v3/users => generated 289 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:01:50.918138 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b7b02d-e859-4655-a228-b657f9cdfa80 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:50.928649 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b7b02d-e859-4655-a228-b657f9cdfa80 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:50.933495 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042d815a-60e5-42bf-b9a1-e6846edc334f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:50.934101 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d815a-60e5-42bf-b9a1-e6846edc334f admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:50.934434 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d815a-60e5-42bf-b9a1-e6846edc334f admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:50.934664 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d815a-60e5-42bf-b9a1-e6846edc334f admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:50.935383 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042d815a-60e5-42bf-b9a1-e6846edc334f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:50.937018 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042d815a-60e5-42bf-b9a1-e6846edc334f admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:50.948150 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1385/2795] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 13:01:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:01:50.952907 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd22e735-8740-4ed0-a6d8-299f2c82ff30 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:50.963852 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd22e735-8740-4ed0-a6d8-299f2c82ff30 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:50.969549 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880718ca-e81c-43fc-9a7c-484bc0cf7edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:50.970262 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880718ca-e81c-43fc-9a7c-484bc0cf7edc admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:50.970592 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880718ca-e81c-43fc-9a7c-484bc0cf7edc admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:50.970878 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880718ca-e81c-43fc-9a7c-484bc0cf7edc admin admin] PATH_INFO: `/v3/projects/eb72505c724c49948513f9568abf60a5/users/a4b13ef764ea431a8a0843022ac23872/roles/0e78444e43c744668178971af4087ca0` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:50.987436 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880718ca-e81c-43fc-9a7c-484bc0cf7edc admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb72505c724c49948513f9568abf60a5, user_id=a4b13ef764ea431a8a0843022ac23872, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:50.990531 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880718ca-e81c-43fc-9a7c-484bc0cf7edc admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:51.004277 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1411/2796] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 13:01:50 2022] PUT /identity/v3/projects/eb72505c724c49948513f9568abf60a5/users/a4b13ef764ea431a8a0843022ac23872/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:01:51.008389 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042d815a-60e5-42bf-b9a1-e6846edc334f admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:51.019578 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042d815a-60e5-42bf-b9a1-e6846edc334f admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:51.079377 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3032168e-180e-44b9-a608-6abd23c8b3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:51.080023 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032168e-180e-44b9-a608-6abd23c8b3f3 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:51.080309 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032168e-180e-44b9-a608-6abd23c8b3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:51.080586 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032168e-180e-44b9-a608-6abd23c8b3f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:51.081292 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3032168e-180e-44b9-a608-6abd23c8b3f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:51.082884 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3032168e-180e-44b9-a608-6abd23c8b3f3 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:51.093194 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1386/2797] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 13:01:51 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:01:51.097389 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880718ca-e81c-43fc-9a7c-484bc0cf7edc admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:51.107693 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880718ca-e81c-43fc-9a7c-484bc0cf7edc admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:51.112406 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b662878b-bc4a-4206-815b-e3ac56bd60d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:51.113060 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662878b-bc4a-4206-815b-e3ac56bd60d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:51.113343 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662878b-bc4a-4206-815b-e3ac56bd60d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:51.113621 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662878b-bc4a-4206-815b-e3ac56bd60d0 admin admin] PATH_INFO: `/v3/projects/eb72505c724c49948513f9568abf60a5/users/a4b13ef764ea431a8a0843022ac23872/roles/0e78444e43c744668178971af4087ca0` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:51.115576 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b662878b-bc4a-4206-815b-e3ac56bd60d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb72505c724c49948513f9568abf60a5, user_id=a4b13ef764ea431a8a0843022ac23872, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:51.118262 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b662878b-bc4a-4206-815b-e3ac56bd60d0 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:51.128893 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1412/2798] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 13:01:51 2022] PUT /identity/v3/projects/eb72505c724c49948513f9568abf60a5/users/a4b13ef764ea431a8a0843022ac23872/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:01:51.139953 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9cc60-c242-470f-8d3f-d9bd30c281d0 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:51.140244 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9cc60-c242-470f-8d3f-d9bd30c281d0 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:51.140526 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9cc60-c242-470f-8d3f-d9bd30c281d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:51.170158 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-06d9cc60-c242-470f-8d3f-d9bd30c281d0 None None] MFA Rules not processed for user `a4b13ef764ea431a8a0843022ac23872`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:01:51.171375 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-06d9cc60-c242-470f-8d3f-d9bd30c281d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:51.246222 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:01:51.258459 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1387/2799] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 13:01:51 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:01:51.270772 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b662878b-bc4a-4206-815b-e3ac56bd60d0 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:51.272479 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-b662878b-bc4a-4206-815b-e3ac56bd60d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:51.298951 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:01:51.332400 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b662878b-bc4a-4206-815b-e3ac56bd60d0 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:51.336077 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22777a7-9e3b-4bf3-a9b1-34b8238527c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:51.336685 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22777a7-9e3b-4bf3-a9b1-34b8238527c2 service cinder] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:51.336965 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22777a7-9e3b-4bf3-a9b1-34b8238527c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:51.337241 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22777a7-9e3b-4bf3-a9b1-34b8238527c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:51.346357 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22777a7-9e3b-4bf3-a9b1-34b8238527c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:51.347810 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22777a7-9e3b-4bf3-a9b1-34b8238527c2 service cinder] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:51.358034 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1413/2800] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 13:01:51 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:01:52.720968 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d9cc60-c242-470f-8d3f-d9bd30c281d0 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:52.782686 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d9cc60-c242-470f-8d3f-d9bd30c281d0 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:52.787189 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a54dc18-2eeb-4340-992d-f119d1b5292d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:52.787850 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54dc18-2eeb-4340-992d-f119d1b5292d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:52.788130 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54dc18-2eeb-4340-992d-f119d1b5292d admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:52.788404 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54dc18-2eeb-4340-992d-f119d1b5292d admin admin] PATH_INFO: `/v3/users/53d22983e68b48bb9d3b87cde8b79904` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:52.789568 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a54dc18-2eeb-4340-992d-f119d1b5292d admin admin] RBAC: Authorizing `identity:delete_user(user_id=53d22983e68b48bb9d3b87cde8b79904)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:52.791525 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a54dc18-2eeb-4340-992d-f119d1b5292d admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:52.838671 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-9a54dc18-2eeb-4340-992d-f119d1b5292d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53d22983e68b48bb9d3b87cde8b79904'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:52.839301 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-9a54dc18-2eeb-4340-992d-f119d1b5292d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53d22983e68b48bb9d3b87cde8b79904'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:52.839925 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-9a54dc18-2eeb-4340-992d-f119d1b5292d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53d22983e68b48bb9d3b87cde8b79904'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:52.851949 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-9a54dc18-2eeb-4340-992d-f119d1b5292d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53d22983e68b48bb9d3b87cde8b79904'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:52.859779 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-9a54dc18-2eeb-4340-992d-f119d1b5292d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53d22983e68b48bb9d3b87cde8b79904'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:52.872394 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1388/2801] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 13:01:52 2022] DELETE /identity/v3/users/53d22983e68b48bb9d3b87cde8b79904 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:01:52.877640 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3deb30-9319-4a25-9ce8-cbc3be6a7e65 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:52.878149 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3deb30-9319-4a25-9ce8-cbc3be6a7e65 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:52.878280 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3deb30-9319-4a25-9ce8-cbc3be6a7e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:52.897379 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-6f3deb30-9319-4a25-9ce8-cbc3be6a7e65 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:01:52.898671 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-6f3deb30-9319-4a25-9ce8-cbc3be6a7e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:52.960891 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1414/2802] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 13:01:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:01:52.970654 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a54dc18-2eeb-4340-992d-f119d1b5292d admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:52.972127 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-9a54dc18-2eeb-4340-992d-f119d1b5292d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:53.153992 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a54dc18-2eeb-4340-992d-f119d1b5292d admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:53.157644 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69711a4d-3085-4e19-8486-664f9160e848 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:53.158252 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69711a4d-3085-4e19-8486-664f9160e848 service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:53.158524 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69711a4d-3085-4e19-8486-664f9160e848 service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:53.158792 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69711a4d-3085-4e19-8486-664f9160e848 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:53.168453 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69711a4d-3085-4e19-8486-664f9160e848 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:53.169909 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69711a4d-3085-4e19-8486-664f9160e848 service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:53.180095 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1389/2803] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 13:01:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:01:53.440938 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3deb30-9319-4a25-9ce8-cbc3be6a7e65 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:53.442475 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-6f3deb30-9319-4a25-9ce8-cbc3be6a7e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:53.465244 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3deb30-9319-4a25-9ce8-cbc3be6a7e65 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:53.469941 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef1f4b8-1422-4168-ab3b-ece955bea7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:53.470580 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1f4b8-1422-4168-ab3b-ece955bea7be admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:53.470878 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1f4b8-1422-4168-ab3b-ece955bea7be admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:53.471133 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1f4b8-1422-4168-ab3b-ece955bea7be admin admin] PATH_INFO: `/v3/projects/1cf43929a1b0493c861d4f76d9d29f79` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:53.472292 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef1f4b8-1422-4168-ab3b-ece955bea7be admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cf43929a1b0493c861d4f76d9d29f79)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:53.474214 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef1f4b8-1422-4168-ab3b-ece955bea7be admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:53.521699 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-aef1f4b8-1422-4168-ab3b-ece955bea7be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cf43929a1b0493c861d4f76d9d29f79', 'request_id': 'req-aef1f4b8-1422-4168-ab3b-ece955bea7be'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:53.527854 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-aef1f4b8-1422-4168-ab3b-ece955bea7be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cf43929a1b0493c861d4f76d9d29f79', 'request_id': 'req-aef1f4b8-1422-4168-ab3b-ece955bea7be'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:53.535831 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-aef1f4b8-1422-4168-ab3b-ece955bea7be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cf43929a1b0493c861d4f76d9d29f79', 'request_id': 'req-aef1f4b8-1422-4168-ab3b-ece955bea7be'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:53.546514 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-aef1f4b8-1422-4168-ab3b-ece955bea7be admin admin] The token cache is being invalidate because project 1cf43929a1b0493c861d4f76d9d29f79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:01:53.546806 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-aef1f4b8-1422-4168-ab3b-ece955bea7be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:53.547864 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-aef1f4b8-1422-4168-ab3b-ece955bea7be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:53.549444 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1415/2804] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 13:01:53 2022] DELETE /identity/v3/projects/1cf43929a1b0493c861d4f76d9d29f79 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:01:53.553507 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69711a4d-3085-4e19-8486-664f9160e848 service neutron] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:53.563510 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69711a4d-3085-4e19-8486-664f9160e848 service neutron] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:53.575569 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:01:53.619933 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a76c85e-b3a7-41b1-a3d2-2d53fe9c6f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:53.620559 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a76c85e-b3a7-41b1-a3d2-2d53fe9c6f30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:53.620846 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a76c85e-b3a7-41b1-a3d2-2d53fe9c6f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:53.621097 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a76c85e-b3a7-41b1-a3d2-2d53fe9c6f30 admin admin] PATH_INFO: `/v3/users/a4b13ef764ea431a8a0843022ac23872` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:53.622227 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a76c85e-b3a7-41b1-a3d2-2d53fe9c6f30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4b13ef764ea431a8a0843022ac23872)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:53.624785 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a76c85e-b3a7-41b1-a3d2-2d53fe9c6f30 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:53.669416 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-9a76c85e-b3a7-41b1-a3d2-2d53fe9c6f30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4b13ef764ea431a8a0843022ac23872'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:53.670050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-9a76c85e-b3a7-41b1-a3d2-2d53fe9c6f30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4b13ef764ea431a8a0843022ac23872'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:53.671197 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-9a76c85e-b3a7-41b1-a3d2-2d53fe9c6f30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4b13ef764ea431a8a0843022ac23872'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:53.682700 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-9a76c85e-b3a7-41b1-a3d2-2d53fe9c6f30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4b13ef764ea431a8a0843022ac23872'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:53.690143 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-9a76c85e-b3a7-41b1-a3d2-2d53fe9c6f30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4b13ef764ea431a8a0843022ac23872'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:53.702351 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1390/2805] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 13:01:53 2022] DELETE /identity/v3/users/a4b13ef764ea431a8a0843022ac23872 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:01:53.984559 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef1f4b8-1422-4168-ab3b-ece955bea7be admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:53.987129 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-aef1f4b8-1422-4168-ab3b-ece955bea7be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:54.006730 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef1f4b8-1422-4168-ab3b-ece955bea7be admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:54.055581 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:01:54.069607 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016f07f2-0938-4b9d-8d01-19b6165432e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:54.070251 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f07f2-0938-4b9d-8d01-19b6165432e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:54.070532 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f07f2-0938-4b9d-8d01-19b6165432e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:54.070804 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f07f2-0938-4b9d-8d01-19b6165432e5 admin admin] PATH_INFO: `/v3/projects/eb72505c724c49948513f9568abf60a5` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:54.071999 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016f07f2-0938-4b9d-8d01-19b6165432e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb72505c724c49948513f9568abf60a5)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:54.073938 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016f07f2-0938-4b9d-8d01-19b6165432e5 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:54.119345 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-016f07f2-0938-4b9d-8d01-19b6165432e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb72505c724c49948513f9568abf60a5', 'request_id': 'req-016f07f2-0938-4b9d-8d01-19b6165432e5'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:54.125393 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-016f07f2-0938-4b9d-8d01-19b6165432e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb72505c724c49948513f9568abf60a5', 'request_id': 'req-016f07f2-0938-4b9d-8d01-19b6165432e5'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:54.133009 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-016f07f2-0938-4b9d-8d01-19b6165432e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb72505c724c49948513f9568abf60a5', 'request_id': 'req-016f07f2-0938-4b9d-8d01-19b6165432e5'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:54.143511 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-016f07f2-0938-4b9d-8d01-19b6165432e5 admin admin] The token cache is being invalidate because project eb72505c724c49948513f9568abf60a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:01:54.143825 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-016f07f2-0938-4b9d-8d01-19b6165432e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:54.144453 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-016f07f2-0938-4b9d-8d01-19b6165432e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:54.145945 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1416/2806] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 13:01:53 2022] DELETE /identity/v3/projects/eb72505c724c49948513f9568abf60a5 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:01:54.151124 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2720b49-fd54-4201-8a24-989a2df57253 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:54.151608 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2720b49-fd54-4201-8a24-989a2df57253 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:54.151772 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2720b49-fd54-4201-8a24-989a2df57253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:54.172572 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-b2720b49-fd54-4201-8a24-989a2df57253 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:01:54.173347 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-b2720b49-fd54-4201-8a24-989a2df57253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:54.240099 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1391/2807] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 13:01:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:01:54.248147 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd560d2-b396-4964-87f9-da4aa92763d7 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:54.248442 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd560d2-b396-4964-87f9-da4aa92763d7 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:54.248782 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd560d2-b396-4964-87f9-da4aa92763d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:54.268819 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-6dd560d2-b396-4964-87f9-da4aa92763d7 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:01:54.270031 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-6dd560d2-b396-4964-87f9-da4aa92763d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:54.277945 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1417/2808] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 13:01:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:01:54.282859 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2720b49-fd54-4201-8a24-989a2df57253 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:54.292531 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2720b49-fd54-4201-8a24-989a2df57253 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:54.297676 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce051b3e-7145-403a-9466-fa9265323673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:54.298275 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce051b3e-7145-403a-9466-fa9265323673 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:54.298556 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce051b3e-7145-403a-9466-fa9265323673 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:54.298900 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce051b3e-7145-403a-9466-fa9265323673 admin admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:54.299654 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce051b3e-7145-403a-9466-fa9265323673 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:01:54.299989 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce051b3e-7145-403a-9466-fa9265323673 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:54.301565 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce051b3e-7145-403a-9466-fa9265323673 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:54.313147 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1392/2809] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 13:01:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:01:54.317458 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd560d2-b396-4964-87f9-da4aa92763d7 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:54.327510 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd560d2-b396-4964-87f9-da4aa92763d7 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:54.332207 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3b9091-27db-4a73-a6e0-784350f328d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:54.332825 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b9091-27db-4a73-a6e0-784350f328d4 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:54.333102 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b9091-27db-4a73-a6e0-784350f328d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:54.333376 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b9091-27db-4a73-a6e0-784350f328d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:54.334056 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3b9091-27db-4a73-a6e0-784350f328d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:54.336000 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3b9091-27db-4a73-a6e0-784350f328d4 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:54.364398 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1418/2810] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 13:01:54 2022] POST /identity/v3/projects => generated 401 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:01:54.368658 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce051b3e-7145-403a-9466-fa9265323673 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:54.378423 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce051b3e-7145-403a-9466-fa9265323673 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:54.399081 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:01:54.436615 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe5c26-b425-4589-97f1-3cba42ce4fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:54.437238 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe5c26-b425-4589-97f1-3cba42ce4fbd admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:54.437516 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe5c26-b425-4589-97f1-3cba42ce4fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:54.437784 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe5c26-b425-4589-97f1-3cba42ce4fbd admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:54.438428 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffe5c26-b425-4589-97f1-3cba42ce4fbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:54.440356 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffe5c26-b425-4589-97f1-3cba42ce4fbd admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:54.461717 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1393/2811] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 13:01:54 2022] POST /identity/v3/users => generated 301 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:01:54.465961 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3b9091-27db-4a73-a6e0-784350f328d4 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:54.477008 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3b9091-27db-4a73-a6e0-784350f328d4 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:54.483079 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311383e1-f240-4207-b189-6275daa31093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:54.484217 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311383e1-f240-4207-b189-6275daa31093 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:54.484565 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311383e1-f240-4207-b189-6275daa31093 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:54.484903 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311383e1-f240-4207-b189-6275daa31093 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:54.485779 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311383e1-f240-4207-b189-6275daa31093 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:54.487853 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311383e1-f240-4207-b189-6275daa31093 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:54.498417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1419/2812] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 13:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 13:01:54.502313 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe5c26-b425-4589-97f1-3cba42ce4fbd admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:54.512549 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe5c26-b425-4589-97f1-3cba42ce4fbd admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:54.517572 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07a04ae-ee09-4697-9ad6-7b8bf1b61db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:54.518166 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a04ae-ee09-4697-9ad6-7b8bf1b61db7 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:54.518443 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a04ae-ee09-4697-9ad6-7b8bf1b61db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:54.518713 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a04ae-ee09-4697-9ad6-7b8bf1b61db7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:54.519403 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07a04ae-ee09-4697-9ad6-7b8bf1b61db7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:54.521014 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07a04ae-ee09-4697-9ad6-7b8bf1b61db7 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:54.531521 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1394/2813] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 13:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:01:54.535753 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311383e1-f240-4207-b189-6275daa31093 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:54.546433 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311383e1-f240-4207-b189-6275daa31093 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:54.551443 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca7bc1c-1322-4707-a103-1d8b676fda03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:54.552131 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7bc1c-1322-4707-a103-1d8b676fda03 admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:54.552408 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7bc1c-1322-4707-a103-1d8b676fda03 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:54.552678 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7bc1c-1322-4707-a103-1d8b676fda03 admin admin] PATH_INFO: `/v3/projects/181a29ff82854a2e929406923e2a2515/users/e6a42206fbba4ebaa9d2a586ab74e44f/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:54.569634 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca7bc1c-1322-4707-a103-1d8b676fda03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=181a29ff82854a2e929406923e2a2515, user_id=e6a42206fbba4ebaa9d2a586ab74e44f, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:54.572358 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca7bc1c-1322-4707-a103-1d8b676fda03 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:54.587856 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1420/2814] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 13:01:54 2022] PUT /identity/v3/projects/181a29ff82854a2e929406923e2a2515/users/e6a42206fbba4ebaa9d2a586ab74e44f/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:01:54.598906 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f14ca4a-5cd1-4343-bd85-7b5fe30309f1 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:54.599187 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f14ca4a-5cd1-4343-bd85-7b5fe30309f1 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:54.599458 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f14ca4a-5cd1-4343-bd85-7b5fe30309f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:54.626430 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-0f14ca4a-5cd1-4343-bd85-7b5fe30309f1 None None] MFA Rules not processed for user `e6a42206fbba4ebaa9d2a586ab74e44f`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:01:54.627658 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-0f14ca4a-5cd1-4343-bd85-7b5fe30309f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:54.691370 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1395/2815] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 13:01:54 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:01:54.695890 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca7bc1c-1322-4707-a103-1d8b676fda03 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:54.766426 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca7bc1c-1322-4707-a103-1d8b676fda03 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:54.771113 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a332b778-8a01-4c18-8613-dab01e9b0d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:54.771760 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a332b778-8a01-4c18-8613-dab01e9b0d52 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:54.772043 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a332b778-8a01-4c18-8613-dab01e9b0d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:54.772315 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a332b778-8a01-4c18-8613-dab01e9b0d52 admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:54.772976 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a332b778-8a01-4c18-8613-dab01e9b0d52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:54.774884 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a332b778-8a01-4c18-8613-dab01e9b0d52 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:54.795673 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:01:54.803779 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1421/2816] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 13:01:54 2022] POST /identity/v3/projects => generated 399 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:01:54.807790 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f14ca4a-5cd1-4343-bd85-7b5fe30309f1 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:54.819106 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f14ca4a-5cd1-4343-bd85-7b5fe30309f1 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:54.875402 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a00646-91f2-4085-b4fa-7a6733609388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:54.876240 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a00646-91f2-4085-b4fa-7a6733609388 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:54.876519 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a00646-91f2-4085-b4fa-7a6733609388 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:54.876922 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a00646-91f2-4085-b4fa-7a6733609388 admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:54.877454 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a00646-91f2-4085-b4fa-7a6733609388 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:54.879369 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a00646-91f2-4085-b4fa-7a6733609388 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:54.902813 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1396/2817] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 13:01:54 2022] POST /identity/v3/users => generated 306 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:01:54.906891 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a332b778-8a01-4c18-8613-dab01e9b0d52 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:54.917962 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a332b778-8a01-4c18-8613-dab01e9b0d52 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:54.922580 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e631b-2b80-48ca-868c-72360a07ae6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:54.923187 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e631b-2b80-48ca-868c-72360a07ae6b admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:54.923472 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e631b-2b80-48ca-868c-72360a07ae6b admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:54.923789 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e631b-2b80-48ca-868c-72360a07ae6b admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:54.924486 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3e631b-2b80-48ca-868c-72360a07ae6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:54.926099 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3e631b-2b80-48ca-868c-72360a07ae6b admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:54.937111 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1422/2818] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 13:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:01:54.941337 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a00646-91f2-4085-b4fa-7a6733609388 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:54.951450 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a00646-91f2-4085-b4fa-7a6733609388 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:54.956765 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e73658-0f03-43b5-9d2d-9a6bbb28acff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:54.957412 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e73658-0f03-43b5-9d2d-9a6bbb28acff admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:54.957690 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e73658-0f03-43b5-9d2d-9a6bbb28acff admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:54.957964 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e73658-0f03-43b5-9d2d-9a6bbb28acff admin admin] PATH_INFO: `/v3/projects/b07532cefd60498397b6c062b0984be4/users/f141794cac164579ab4d3bfe28742c7e/roles/0e78444e43c744668178971af4087ca0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:54.972589 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e73658-0f03-43b5-9d2d-9a6bbb28acff admin admin] RBAC: Authorizing `identity:create_grant(project_id=b07532cefd60498397b6c062b0984be4, user_id=f141794cac164579ab4d3bfe28742c7e, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:54.975247 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e73658-0f03-43b5-9d2d-9a6bbb28acff admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:54.983142 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:01:54.990704 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1397/2819] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 13:01:54 2022] PUT /identity/v3/projects/b07532cefd60498397b6c062b0984be4/users/f141794cac164579ab4d3bfe28742c7e/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:01:54.994705 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e631b-2b80-48ca-868c-72360a07ae6b admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:55.005520 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e631b-2b80-48ca-868c-72360a07ae6b admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:55.062079 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91357dc-914a-481c-a7e6-31c005345662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:55.062625 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91357dc-914a-481c-a7e6-31c005345662 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:55.062910 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91357dc-914a-481c-a7e6-31c005345662 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:55.063188 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91357dc-914a-481c-a7e6-31c005345662 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:55.063913 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91357dc-914a-481c-a7e6-31c005345662 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:55.065904 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91357dc-914a-481c-a7e6-31c005345662 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:55.077480 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1423/2820] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 13:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:01:55.081707 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e73658-0f03-43b5-9d2d-9a6bbb28acff admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:55.092017 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e73658-0f03-43b5-9d2d-9a6bbb28acff admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:55.096691 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d950bfcf-4b5c-421c-9d6b-2a2375e2f1f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:55.097339 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950bfcf-4b5c-421c-9d6b-2a2375e2f1f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:55.097622 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950bfcf-4b5c-421c-9d6b-2a2375e2f1f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:55.097901 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950bfcf-4b5c-421c-9d6b-2a2375e2f1f9 admin admin] PATH_INFO: `/v3/projects/b07532cefd60498397b6c062b0984be4/users/f141794cac164579ab4d3bfe28742c7e/roles/0e78444e43c744668178971af4087ca0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:55.099985 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d950bfcf-4b5c-421c-9d6b-2a2375e2f1f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b07532cefd60498397b6c062b0984be4, user_id=f141794cac164579ab4d3bfe28742c7e, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:55.102636 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d950bfcf-4b5c-421c-9d6b-2a2375e2f1f9 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:55.113328 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1398/2821] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 13:01:55 2022] PUT /identity/v3/projects/b07532cefd60498397b6c062b0984be4/users/f141794cac164579ab4d3bfe28742c7e/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:01:55.124974 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c63d24-f1ad-427a-abd5-f07e9a8b287d None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:55.125267 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c63d24-f1ad-427a-abd5-f07e9a8b287d None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:55.125544 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c63d24-f1ad-427a-abd5-f07e9a8b287d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:55.153679 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-68c63d24-f1ad-427a-abd5-f07e9a8b287d None None] MFA Rules not processed for user `f141794cac164579ab4d3bfe28742c7e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:01:55.154909 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-68c63d24-f1ad-427a-abd5-f07e9a8b287d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:55.225090 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1424/2822] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 13:01:55 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:01:55.237403 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d950bfcf-4b5c-421c-9d6b-2a2375e2f1f9 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:55.239296 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-d950bfcf-4b5c-421c-9d6b-2a2375e2f1f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:55.297151 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d950bfcf-4b5c-421c-9d6b-2a2375e2f1f9 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:55.300895 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9df6a6-64c7-4926-a91c-7af587c11cfc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:55.301496 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9df6a6-64c7-4926-a91c-7af587c11cfc service cinder] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:55.301770 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9df6a6-64c7-4926-a91c-7af587c11cfc service cinder] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:55.302044 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9df6a6-64c7-4926-a91c-7af587c11cfc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:55.311810 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9df6a6-64c7-4926-a91c-7af587c11cfc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:55.313315 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9df6a6-64c7-4926-a91c-7af587c11cfc service cinder] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:55.322917 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1399/2823] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 13:01:55 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:01:55.400098 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c63d24-f1ad-427a-abd5-f07e9a8b287d None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:55.409950 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c63d24-f1ad-427a-abd5-f07e9a8b287d None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:55.413604 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31bcb09-f1c3-45bd-8b34-b9c587f2e1ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:55.414208 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31bcb09-f1c3-45bd-8b34-b9c587f2e1ed service cinder] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:55.414492 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31bcb09-f1c3-45bd-8b34-b9c587f2e1ed service cinder] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:55.414766 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31bcb09-f1c3-45bd-8b34-b9c587f2e1ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:55.472807 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31bcb09-f1c3-45bd-8b34-b9c587f2e1ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:55.474233 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31bcb09-f1c3-45bd-8b34-b9c587f2e1ed service cinder] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:55.484094 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1425/2824] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 13:01:55 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:01:58.518572 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9df6a6-64c7-4926-a91c-7af587c11cfc service cinder] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:58.520576 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-1f9df6a6-64c7-4926-a91c-7af587c11cfc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:58.634780 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9df6a6-64c7-4926-a91c-7af587c11cfc service cinder] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:58.641646 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c0846-db11-45c4-b99e-f2a53567e614 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:58.642478 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c0846-db11-45c4-b99e-f2a53567e614 service nova] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:58.642819 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c0846-db11-45c4-b99e-f2a53567e614 service nova] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:58.643162 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c0846-db11-45c4-b99e-f2a53567e614 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:58.645246 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-412c0846-db11-45c4-b99e-f2a53567e614 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:58.662943 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412c0846-db11-45c4-b99e-f2a53567e614 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:58.664912 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412c0846-db11-45c4-b99e-f2a53567e614 service nova] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:58.685718 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1400/2825] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 13:01:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:01:59.748140 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31bcb09-f1c3-45bd-8b34-b9c587f2e1ed service cinder] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:59.757756 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:01:59.802989 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c0846-db11-45c4-b99e-f2a53567e614 service nova] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:01:59.805200 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-412c0846-db11-45c4-b99e-f2a53567e614 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:59.819695 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31bcb09-f1c3-45bd-8b34-b9c587f2e1ed service cinder] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:59.824511 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95b3f7-255d-430b-a824-fab02c76dcca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:59.824875 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95b3f7-255d-430b-a824-fab02c76dcca admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:59.825188 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95b3f7-255d-430b-a824-fab02c76dcca admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:59.825463 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95b3f7-255d-430b-a824-fab02c76dcca admin admin] PATH_INFO: `/v3/users/e6a42206fbba4ebaa9d2a586ab74e44f` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:59.827112 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e95b3f7-255d-430b-a824-fab02c76dcca admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6a42206fbba4ebaa9d2a586ab74e44f)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:59.829075 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e95b3f7-255d-430b-a824-fab02c76dcca admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:59.865515 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c0846-db11-45c4-b99e-f2a53567e614 service nova] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:01:59.869622 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ad9db-ac72-4460-ad88-cc2449a8e6a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:01:59.870373 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ad9db-ac72-4460-ad88-cc2449a8e6a1 service neutron] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:59.870655 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ad9db-ac72-4460-ad88-cc2449a8e6a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:59.871135 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ad9db-ac72-4460-ad88-cc2449a8e6a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:59.873119 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-257ad9db-ac72-4460-ad88-cc2449a8e6a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:59.881421 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-6e95b3f7-255d-430b-a824-fab02c76dcca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6a42206fbba4ebaa9d2a586ab74e44f'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:59.884087 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257ad9db-ac72-4460-ad88-cc2449a8e6a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:01:59.885840 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257ad9db-ac72-4460-ad88-cc2449a8e6a1 service neutron] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:01:59.892537 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-6e95b3f7-255d-430b-a824-fab02c76dcca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6a42206fbba4ebaa9d2a586ab74e44f'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:59.893341 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-6e95b3f7-255d-430b-a824-fab02c76dcca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6a42206fbba4ebaa9d2a586ab74e44f'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:59.896223 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1401/2826] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 13:01:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 13:01:59.901653 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-6e95b3f7-255d-430b-a824-fab02c76dcca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6a42206fbba4ebaa9d2a586ab74e44f'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:59.918665 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-6e95b3f7-255d-430b-a824-fab02c76dcca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6a42206fbba4ebaa9d2a586ab74e44f'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:01:59.924982 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1426/2827] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 13:01:59 2022] DELETE /identity/v3/users/e6a42206fbba4ebaa9d2a586ab74e44f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:01:59.929608 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f374b5a-0759-4d58-9428-ed1494f7f482 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:01:59.929891 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f374b5a-0759-4d58-9428-ed1494f7f482 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:01:59.930167 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f374b5a-0759-4d58-9428-ed1494f7f482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:01:59.950360 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-2f374b5a-0759-4d58-9428-ed1494f7f482 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:01:59.952204 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-2f374b5a-0759-4d58-9428-ed1494f7f482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:01:59.962480 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:02:00.015420 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1402/2828] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 13:01:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:02:00.025262 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95b3f7-255d-430b-a824-fab02c76dcca admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:00.026800 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-6e95b3f7-255d-430b-a824-fab02c76dcca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:02:00.088196 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95b3f7-255d-430b-a824-fab02c76dcca admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:00.092582 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185ca97-852f-44e3-9a0c-3b5344d95128 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:00.093219 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185ca97-852f-44e3-9a0c-3b5344d95128 service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:00.093530 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185ca97-852f-44e3-9a0c-3b5344d95128 service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:00.093806 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185ca97-852f-44e3-9a0c-3b5344d95128 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:00.104462 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1185ca97-852f-44e3-9a0c-3b5344d95128 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:00.105938 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1185ca97-852f-44e3-9a0c-3b5344d95128 service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:00.119937 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1427/2829] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 13:02:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:02:00.590092 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f374b5a-0759-4d58-9428-ed1494f7f482 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:00.591532 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-2f374b5a-0759-4d58-9428-ed1494f7f482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:02:00.604435 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f374b5a-0759-4d58-9428-ed1494f7f482 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:00.611922 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a280fcc-4c2f-41d5-9ccd-3001076c2590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:00.612672 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a280fcc-4c2f-41d5-9ccd-3001076c2590 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:00.613029 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a280fcc-4c2f-41d5-9ccd-3001076c2590 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:00.613377 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a280fcc-4c2f-41d5-9ccd-3001076c2590 admin admin] PATH_INFO: `/v3/projects/181a29ff82854a2e929406923e2a2515` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:00.614929 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a280fcc-4c2f-41d5-9ccd-3001076c2590 admin admin] RBAC: Authorizing `identity:delete_project(project_id=181a29ff82854a2e929406923e2a2515)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:00.616935 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a280fcc-4c2f-41d5-9ccd-3001076c2590 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:00.671163 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-5a280fcc-4c2f-41d5-9ccd-3001076c2590 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '181a29ff82854a2e929406923e2a2515', 'request_id': 'req-5a280fcc-4c2f-41d5-9ccd-3001076c2590'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:00.677814 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-5a280fcc-4c2f-41d5-9ccd-3001076c2590 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '181a29ff82854a2e929406923e2a2515', 'request_id': 'req-5a280fcc-4c2f-41d5-9ccd-3001076c2590'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:00.685591 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-5a280fcc-4c2f-41d5-9ccd-3001076c2590 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '181a29ff82854a2e929406923e2a2515', 'request_id': 'req-5a280fcc-4c2f-41d5-9ccd-3001076c2590'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:00.695949 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-5a280fcc-4c2f-41d5-9ccd-3001076c2590 admin admin] The token cache is being invalidate because project 181a29ff82854a2e929406923e2a2515 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:02:00.696239 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-5a280fcc-4c2f-41d5-9ccd-3001076c2590 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:00.696871 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-5a280fcc-4c2f-41d5-9ccd-3001076c2590 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:00.698603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1403/2830] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 13:02:00 2022] DELETE /identity/v3/projects/181a29ff82854a2e929406923e2a2515 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:02:00.702842 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185ca97-852f-44e3-9a0c-3b5344d95128 service neutron] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:00.712657 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185ca97-852f-44e3-9a0c-3b5344d95128 service neutron] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:00.744935 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:02:00.781505 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7b60bc-1670-4d3e-a4af-e17fe2f8c38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:00.782218 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b60bc-1670-4d3e-a4af-e17fe2f8c38b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:00.782505 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b60bc-1670-4d3e-a4af-e17fe2f8c38b admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:00.782785 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b60bc-1670-4d3e-a4af-e17fe2f8c38b admin admin] PATH_INFO: `/v3/users/f141794cac164579ab4d3bfe28742c7e` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:00.784009 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7b60bc-1670-4d3e-a4af-e17fe2f8c38b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f141794cac164579ab4d3bfe28742c7e)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:00.785956 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7b60bc-1670-4d3e-a4af-e17fe2f8c38b admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:00.865664 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-7d7b60bc-1670-4d3e-a4af-e17fe2f8c38b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f141794cac164579ab4d3bfe28742c7e'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:00.876717 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a280fcc-4c2f-41d5-9ccd-3001076c2590 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:00.879135 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-5a280fcc-4c2f-41d5-9ccd-3001076c2590 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:02:00.879282 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-7d7b60bc-1670-4d3e-a4af-e17fe2f8c38b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f141794cac164579ab4d3bfe28742c7e'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:00.881264 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-7d7b60bc-1670-4d3e-a4af-e17fe2f8c38b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f141794cac164579ab4d3bfe28742c7e'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:00.897242 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-7d7b60bc-1670-4d3e-a4af-e17fe2f8c38b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f141794cac164579ab4d3bfe28742c7e'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:00.927938 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-7d7b60bc-1670-4d3e-a4af-e17fe2f8c38b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f141794cac164579ab4d3bfe28742c7e'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:00.936866 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1428/2831] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 13:02:00 2022] DELETE /identity/v3/users/f141794cac164579ab4d3bfe28742c7e => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:02:00.993063 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a280fcc-4c2f-41d5-9ccd-3001076c2590 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:01.032301 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:02:01.066621 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb634f2-f74f-4b5c-840c-2794819f490c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5d6f3c12244c0faf26048513dc49e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:01.067361 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb634f2-f74f-4b5c-840c-2794819f490c service nova] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:01.067752 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb634f2-f74f-4b5c-840c-2794819f490c service nova] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:01.068106 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb634f2-f74f-4b5c-840c-2794819f490c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:01.070552 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-0fb634f2-f74f-4b5c-840c-2794819f490c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:02:01.083490 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb634f2-f74f-4b5c-840c-2794819f490c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:01.085122 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb634f2-f74f-4b5c-840c-2794819f490c service nova] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:01.097488 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1404/2832] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 13:02:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 13:02:01.269488 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7b60bc-1670-4d3e-a4af-e17fe2f8c38b admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:01.271284 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-7d7b60bc-1670-4d3e-a4af-e17fe2f8c38b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:02:01.343088 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7b60bc-1670-4d3e-a4af-e17fe2f8c38b admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:01.347935 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02920e48-2f06-47b4-9e9a-a67ea363a9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:01.348670 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02920e48-2f06-47b4-9e9a-a67ea363a9aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:01.348952 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02920e48-2f06-47b4-9e9a-a67ea363a9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:01.349228 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02920e48-2f06-47b4-9e9a-a67ea363a9aa admin admin] PATH_INFO: `/v3/projects/b07532cefd60498397b6c062b0984be4` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:01.350572 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02920e48-2f06-47b4-9e9a-a67ea363a9aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=b07532cefd60498397b6c062b0984be4)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:01.352786 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02920e48-2f06-47b4-9e9a-a67ea363a9aa admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:01.400144 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-02920e48-2f06-47b4-9e9a-a67ea363a9aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b07532cefd60498397b6c062b0984be4', 'request_id': 'req-02920e48-2f06-47b4-9e9a-a67ea363a9aa'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:01.401924 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:02:01.408167 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-02920e48-2f06-47b4-9e9a-a67ea363a9aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b07532cefd60498397b6c062b0984be4', 'request_id': 'req-02920e48-2f06-47b4-9e9a-a67ea363a9aa'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:01.416490 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-02920e48-2f06-47b4-9e9a-a67ea363a9aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b07532cefd60498397b6c062b0984be4', 'request_id': 'req-02920e48-2f06-47b4-9e9a-a67ea363a9aa'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:01.426830 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-02920e48-2f06-47b4-9e9a-a67ea363a9aa admin admin] The token cache is being invalidate because project b07532cefd60498397b6c062b0984be4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:02:01.427110 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-02920e48-2f06-47b4-9e9a-a67ea363a9aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:01.427780 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-02920e48-2f06-47b4-9e9a-a67ea363a9aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:01.429293 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1429/2833] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 13:02:01 2022] DELETE /identity/v3/projects/b07532cefd60498397b6c062b0984be4 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:02:01.434126 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c910091-3671-474f-8ac6-c3ae57fb0c21 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:01.434411 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c910091-3671-474f-8ac6-c3ae57fb0c21 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:01.434685 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c910091-3671-474f-8ac6-c3ae57fb0c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:01.454427 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-0c910091-3671-474f-8ac6-c3ae57fb0c21 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:02:01.455614 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-0c910091-3671-474f-8ac6-c3ae57fb0c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:02:01.519322 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1405/2834] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 13:02:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:02:01.527101 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0d7b2-25d1-41e6-8cb1-1db4f4c77b4e None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:01.527615 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0d7b2-25d1-41e6-8cb1-1db4f4c77b4e None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:01.527927 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0d7b2-25d1-41e6-8cb1-1db4f4c77b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:01.548363 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-00f0d7b2-25d1-41e6-8cb1-1db4f4c77b4e None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:02:01.549581 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-00f0d7b2-25d1-41e6-8cb1-1db4f4c77b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:02:01.559132 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1430/2835] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 13:02:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:02:01.563151 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c910091-3671-474f-8ac6-c3ae57fb0c21 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:01.574068 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c910091-3671-474f-8ac6-c3ae57fb0c21 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:01.579886 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e61fb4d-636a-4127-bf22-35c99416fbc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:01.580497 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61fb4d-636a-4127-bf22-35c99416fbc7 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:01.580779 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61fb4d-636a-4127-bf22-35c99416fbc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:01.581834 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61fb4d-636a-4127-bf22-35c99416fbc7 admin admin] PATH_INFO: `/v3/domains` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:01.582731 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e61fb4d-636a-4127-bf22-35c99416fbc7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:02:01.582924 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e61fb4d-636a-4127-bf22-35c99416fbc7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:01.584997 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e61fb4d-636a-4127-bf22-35c99416fbc7 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:01.597585 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1406/2836] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 13:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:02:01.602427 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f0d7b2-25d1-41e6-8cb1-1db4f4c77b4e None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:01.613039 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f0d7b2-25d1-41e6-8cb1-1db4f4c77b4e None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:01.618008 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f392b80-1914-4242-b34a-6dbbfa05c7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:01.618627 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f392b80-1914-4242-b34a-6dbbfa05c7d8 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:01.619789 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f392b80-1914-4242-b34a-6dbbfa05c7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:01.620101 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f392b80-1914-4242-b34a-6dbbfa05c7d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:01.620810 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f392b80-1914-4242-b34a-6dbbfa05c7d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:01.622762 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f392b80-1914-4242-b34a-6dbbfa05c7d8 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:01.652022 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1431/2837] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 13:02:01 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:02:01.656612 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e61fb4d-636a-4127-bf22-35c99416fbc7 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:01.668923 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e61fb4d-636a-4127-bf22-35c99416fbc7 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:01.725749 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee339b0-2108-41bb-a05e-d7ca4b0f1efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:01.726359 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee339b0-2108-41bb-a05e-d7ca4b0f1efa admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:01.726641 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee339b0-2108-41bb-a05e-d7ca4b0f1efa admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:01.726921 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee339b0-2108-41bb-a05e-d7ca4b0f1efa admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:01.727570 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee339b0-2108-41bb-a05e-d7ca4b0f1efa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:01.729554 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee339b0-2108-41bb-a05e-d7ca4b0f1efa admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:01.750800 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1407/2838] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 13:02:01 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:02:01.754944 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f392b80-1914-4242-b34a-6dbbfa05c7d8 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:01.768261 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f392b80-1914-4242-b34a-6dbbfa05c7d8 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:01.773128 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c3a74-0b5d-4a84-9477-9e9017ca4dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:01.773769 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c3a74-0b5d-4a84-9477-9e9017ca4dd2 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:01.774045 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c3a74-0b5d-4a84-9477-9e9017ca4dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:01.774315 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c3a74-0b5d-4a84-9477-9e9017ca4dd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:01.775004 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074c3a74-0b5d-4a84-9477-9e9017ca4dd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:01.776679 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074c3a74-0b5d-4a84-9477-9e9017ca4dd2 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:01.787447 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1432/2839] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 13:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:02:01.791522 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee339b0-2108-41bb-a05e-d7ca4b0f1efa admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:01.803994 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee339b0-2108-41bb-a05e-d7ca4b0f1efa admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:01.808669 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2fe2db-a034-4493-8974-27cdf88cc679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:01.809318 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2fe2db-a034-4493-8974-27cdf88cc679 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:01.809597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2fe2db-a034-4493-8974-27cdf88cc679 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:01.809872 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2fe2db-a034-4493-8974-27cdf88cc679 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:01.810610 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2fe2db-a034-4493-8974-27cdf88cc679 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:01.812794 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2fe2db-a034-4493-8974-27cdf88cc679 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:01.824487 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1408/2840] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 13:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 13:02:01.829455 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c3a74-0b5d-4a84-9477-9e9017ca4dd2 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:01.839396 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c3a74-0b5d-4a84-9477-9e9017ca4dd2 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:01.844329 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479eec40-33ab-4cc0-a294-bc4d0c4196c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:01.844988 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479eec40-33ab-4cc0-a294-bc4d0c4196c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:01.845268 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479eec40-33ab-4cc0-a294-bc4d0c4196c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:01.845543 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479eec40-33ab-4cc0-a294-bc4d0c4196c5 admin admin] PATH_INFO: `/v3/projects/05f090510c674256a962928a1cb5ee15/users/9e4d2a9d79cd4dabb5d3b647003715e6/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:01.860626 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479eec40-33ab-4cc0-a294-bc4d0c4196c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05f090510c674256a962928a1cb5ee15, user_id=9e4d2a9d79cd4dabb5d3b647003715e6, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:01.863295 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479eec40-33ab-4cc0-a294-bc4d0c4196c5 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:01.876323 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1433/2841] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 13:02:01 2022] PUT /identity/v3/projects/05f090510c674256a962928a1cb5ee15/users/9e4d2a9d79cd4dabb5d3b647003715e6/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:02:01.887050 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0611ae8-a5b2-4cb5-b431-06a45fe0db72 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:01.887526 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0611ae8-a5b2-4cb5-b431-06a45fe0db72 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:01.887626 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0611ae8-a5b2-4cb5-b431-06a45fe0db72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:01.915683 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-d0611ae8-a5b2-4cb5-b431-06a45fe0db72 None None] MFA Rules not processed for user `9e4d2a9d79cd4dabb5d3b647003715e6`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:02:01.916988 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-d0611ae8-a5b2-4cb5-b431-06a45fe0db72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:02:01.933591 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:02:01.988184 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1409/2842] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 13:02:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:02:01.992760 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479eec40-33ab-4cc0-a294-bc4d0c4196c5 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:02.054418 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479eec40-33ab-4cc0-a294-bc4d0c4196c5 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:02.059110 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae4761-9414-4f3e-ad57-dfa312fb651b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:02.059754 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae4761-9414-4f3e-ad57-dfa312fb651b admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:02.060040 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae4761-9414-4f3e-ad57-dfa312fb651b admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:02.060373 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae4761-9414-4f3e-ad57-dfa312fb651b admin admin] PATH_INFO: `/v3/projects` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:02.061623 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bae4761-9414-4f3e-ad57-dfa312fb651b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:02.063526 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bae4761-9414-4f3e-ad57-dfa312fb651b admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:02.095566 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1434/2843] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 13:02:01 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 13:02:02.100629 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0611ae8-a5b2-4cb5-b431-06a45fe0db72 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:02.112216 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0611ae8-a5b2-4cb5-b431-06a45fe0db72 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:02.168432 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f484c8-6297-4931-8778-c45391a90ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:02.168959 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f484c8-6297-4931-8778-c45391a90ecd admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:02.169234 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f484c8-6297-4931-8778-c45391a90ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:02.169504 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f484c8-6297-4931-8778-c45391a90ecd admin admin] PATH_INFO: `/v3/users` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:02.170724 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f484c8-6297-4931-8778-c45391a90ecd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:02.172671 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f484c8-6297-4931-8778-c45391a90ecd admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:02.194101 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1410/2844] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 13:02:02 2022] POST /identity/v3/users => generated 297 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 13:02:02.198098 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae4761-9414-4f3e-ad57-dfa312fb651b admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:02.208541 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae4761-9414-4f3e-ad57-dfa312fb651b admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:02.213037 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dcbd6a-cfa2-4184-950d-8aae868c18d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:02.213634 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dcbd6a-cfa2-4184-950d-8aae868c18d4 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:02.213912 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dcbd6a-cfa2-4184-950d-8aae868c18d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:02.214182 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dcbd6a-cfa2-4184-950d-8aae868c18d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:02.214864 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dcbd6a-cfa2-4184-950d-8aae868c18d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:02.216493 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dcbd6a-cfa2-4184-950d-8aae868c18d4 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:02.229415 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1435/2845] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 13:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:02:02.234309 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f484c8-6297-4931-8778-c45391a90ecd admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:02.244440 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f484c8-6297-4931-8778-c45391a90ecd admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:02.249206 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a5580-a148-4f82-81ca-50d3601dc0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:02.249858 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a5580-a148-4f82-81ca-50d3601dc0d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:02.250139 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a5580-a148-4f82-81ca-50d3601dc0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:02.250416 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a5580-a148-4f82-81ca-50d3601dc0d2 admin admin] PATH_INFO: `/v3/projects/a6f59bc5df02426983e0e8fe2a4bfa93/users/6a317c1c88d449ca9c7109b4cf12a2e8/roles/0e78444e43c744668178971af4087ca0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:02.267157 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438a5580-a148-4f82-81ca-50d3601dc0d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6f59bc5df02426983e0e8fe2a4bfa93, user_id=6a317c1c88d449ca9c7109b4cf12a2e8, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:02.269880 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438a5580-a148-4f82-81ca-50d3601dc0d2 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:02.284956 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1411/2846] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 13:02:02 2022] PUT /identity/v3/projects/a6f59bc5df02426983e0e8fe2a4bfa93/users/6a317c1c88d449ca9c7109b4cf12a2e8/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:02:02.289125 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dcbd6a-cfa2-4184-950d-8aae868c18d4 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:02.301600 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dcbd6a-cfa2-4184-950d-8aae868c18d4 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:02.351763 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:02:02.364361 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f7e6c2-0945-459e-b884-1c346af944c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:02.365861 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f7e6c2-0945-459e-b884-1c346af944c9 admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:02.366139 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f7e6c2-0945-459e-b884-1c346af944c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:02.366413 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f7e6c2-0945-459e-b884-1c346af944c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:02.367162 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f7e6c2-0945-459e-b884-1c346af944c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:02.369040 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f7e6c2-0945-459e-b884-1c346af944c9 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:02.379512 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1436/2847] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 13:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:02:02.384300 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a5580-a148-4f82-81ca-50d3601dc0d2 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:02.395821 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a5580-a148-4f82-81ca-50d3601dc0d2 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:02.400795 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd066ed6-a870-499d-928f-d869bddcab7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:02.401657 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd066ed6-a870-499d-928f-d869bddcab7a admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:02.401934 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd066ed6-a870-499d-928f-d869bddcab7a admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:02.402203 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd066ed6-a870-499d-928f-d869bddcab7a admin admin] PATH_INFO: `/v3/projects/a6f59bc5df02426983e0e8fe2a4bfa93/users/6a317c1c88d449ca9c7109b4cf12a2e8/roles/0e78444e43c744668178971af4087ca0` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:02.404253 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd066ed6-a870-499d-928f-d869bddcab7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6f59bc5df02426983e0e8fe2a4bfa93, user_id=6a317c1c88d449ca9c7109b4cf12a2e8, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:02.407490 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd066ed6-a870-499d-928f-d869bddcab7a admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:02.417883 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1412/2848] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 13:02:02 2022] PUT /identity/v3/projects/a6f59bc5df02426983e0e8fe2a4bfa93/users/6a317c1c88d449ca9c7109b4cf12a2e8/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:02:02.429662 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8295ae-bd13-4f8a-9092-f1dc42dc0e31 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:02.429910 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8295ae-bd13-4f8a-9092-f1dc42dc0e31 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:02.430181 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8295ae-bd13-4f8a-9092-f1dc42dc0e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:02.456977 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-0b8295ae-bd13-4f8a-9092-f1dc42dc0e31 None None] MFA Rules not processed for user `6a317c1c88d449ca9c7109b4cf12a2e8`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:02:02.458169 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-0b8295ae-bd13-4f8a-9092-f1dc42dc0e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:02:02.530804 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1437/2849] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 13:02:02 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:02:02.541462 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd066ed6-a870-499d-928f-d869bddcab7a admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:02.543174 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-bd066ed6-a870-499d-928f-d869bddcab7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:02:02.605167 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd066ed6-a870-499d-928f-d869bddcab7a admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:02.608897 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648419d7-7374-4523-9f43-06acba66517b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:02.609512 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648419d7-7374-4523-9f43-06acba66517b service cinder] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:02.609799 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648419d7-7374-4523-9f43-06acba66517b service cinder] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:02.610064 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648419d7-7374-4523-9f43-06acba66517b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:02.619797 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648419d7-7374-4523-9f43-06acba66517b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:02.621273 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648419d7-7374-4523-9f43-06acba66517b service cinder] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:02.630701 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1413/2850] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 13:02:02 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:02:03.006056 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8295ae-bd13-4f8a-9092-f1dc42dc0e31 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:03.067625 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8295ae-bd13-4f8a-9092-f1dc42dc0e31 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:03.072765 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c96024-2333-4d90-b365-f6a5af4589b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:03.073433 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c96024-2333-4d90-b365-f6a5af4589b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:03.073713 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c96024-2333-4d90-b365-f6a5af4589b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:03.073986 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c96024-2333-4d90-b365-f6a5af4589b8 admin admin] PATH_INFO: `/v3/users/9e4d2a9d79cd4dabb5d3b647003715e6` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:03.075146 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c96024-2333-4d90-b365-f6a5af4589b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e4d2a9d79cd4dabb5d3b647003715e6)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:03.077139 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c96024-2333-4d90-b365-f6a5af4589b8 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:03.123774 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-90c96024-2333-4d90-b365-f6a5af4589b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e4d2a9d79cd4dabb5d3b647003715e6'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:03.131022 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-90c96024-2333-4d90-b365-f6a5af4589b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e4d2a9d79cd4dabb5d3b647003715e6'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:03.131716 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-90c96024-2333-4d90-b365-f6a5af4589b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e4d2a9d79cd4dabb5d3b647003715e6'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:03.139325 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-90c96024-2333-4d90-b365-f6a5af4589b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e4d2a9d79cd4dabb5d3b647003715e6'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:03.147023 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:02:03.151888 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-90c96024-2333-4d90-b365-f6a5af4589b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e4d2a9d79cd4dabb5d3b647003715e6'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:03.159207 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1438/2851] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 13:02:03 2022] DELETE /identity/v3/users/9e4d2a9d79cd4dabb5d3b647003715e6 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:02:03.163738 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dbbbc5-5254-4b2a-910e-180889e1633b None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:03.164226 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dbbbc5-5254-4b2a-910e-180889e1633b None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:03.164347 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dbbbc5-5254-4b2a-910e-180889e1633b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:03.184336 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-13dbbbc5-5254-4b2a-910e-180889e1633b None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:02:03.185521 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-13dbbbc5-5254-4b2a-910e-180889e1633b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:02:03.194492 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:02:03.252265 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1414/2852] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 13:02:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:02:03.262345 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c96024-2333-4d90-b365-f6a5af4589b8 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:03.263813 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-90c96024-2333-4d90-b365-f6a5af4589b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:02:03.321812 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c96024-2333-4d90-b365-f6a5af4589b8 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:03.325547 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1b64ba-1918-4c21-b56f-a839bddd6956 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:03.326156 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b64ba-1918-4c21-b56f-a839bddd6956 service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:03.326928 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b64ba-1918-4c21-b56f-a839bddd6956 service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:03.327200 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b64ba-1918-4c21-b56f-a839bddd6956 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:03.336268 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1b64ba-1918-4c21-b56f-a839bddd6956 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:03.337694 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1b64ba-1918-4c21-b56f-a839bddd6956 service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:03.347687 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1439/2853] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 13:02:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:02:03.627776 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dbbbc5-5254-4b2a-910e-180889e1633b None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:03.629196 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-13dbbbc5-5254-4b2a-910e-180889e1633b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:02:03.645340 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dbbbc5-5254-4b2a-910e-180889e1633b None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:03.649998 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef8601-a13a-4802-8fbd-8ce3ace65681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:03.650638 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef8601-a13a-4802-8fbd-8ce3ace65681 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:03.650918 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef8601-a13a-4802-8fbd-8ce3ace65681 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:03.651189 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef8601-a13a-4802-8fbd-8ce3ace65681 admin admin] PATH_INFO: `/v3/projects/05f090510c674256a962928a1cb5ee15` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:03.652365 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef8601-a13a-4802-8fbd-8ce3ace65681 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05f090510c674256a962928a1cb5ee15)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:03.654312 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef8601-a13a-4802-8fbd-8ce3ace65681 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:03.700294 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-c0ef8601-a13a-4802-8fbd-8ce3ace65681 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05f090510c674256a962928a1cb5ee15', 'request_id': 'req-c0ef8601-a13a-4802-8fbd-8ce3ace65681'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:03.706875 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-c0ef8601-a13a-4802-8fbd-8ce3ace65681 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05f090510c674256a962928a1cb5ee15', 'request_id': 'req-c0ef8601-a13a-4802-8fbd-8ce3ace65681'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:03.715519 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-c0ef8601-a13a-4802-8fbd-8ce3ace65681 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05f090510c674256a962928a1cb5ee15', 'request_id': 'req-c0ef8601-a13a-4802-8fbd-8ce3ace65681'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:03.727715 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-c0ef8601-a13a-4802-8fbd-8ce3ace65681 admin admin] The token cache is being invalidate because project 05f090510c674256a962928a1cb5ee15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:02:03.728006 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-c0ef8601-a13a-4802-8fbd-8ce3ace65681 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:03.728649 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-c0ef8601-a13a-4802-8fbd-8ce3ace65681 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:03.730168 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1415/2854] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 13:02:03 2022] DELETE /identity/v3/projects/05f090510c674256a962928a1cb5ee15 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:02:03.734389 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1b64ba-1918-4c21-b56f-a839bddd6956 service neutron] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:03.745224 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1b64ba-1918-4c21-b56f-a839bddd6956 service neutron] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:03.806368 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f0fe3e-05ad-4db1-a44b-da41bbb7034f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:03.807071 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f0fe3e-05ad-4db1-a44b-da41bbb7034f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:03.807417 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f0fe3e-05ad-4db1-a44b-da41bbb7034f admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:03.807801 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f0fe3e-05ad-4db1-a44b-da41bbb7034f admin admin] PATH_INFO: `/v3/users/6a317c1c88d449ca9c7109b4cf12a2e8` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:03.809289 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f0fe3e-05ad-4db1-a44b-da41bbb7034f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a317c1c88d449ca9c7109b4cf12a2e8)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:03.813876 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f0fe3e-05ad-4db1-a44b-da41bbb7034f admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:03.867882 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-d1f0fe3e-05ad-4db1-a44b-da41bbb7034f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a317c1c88d449ca9c7109b4cf12a2e8'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:03.876052 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-d1f0fe3e-05ad-4db1-a44b-da41bbb7034f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a317c1c88d449ca9c7109b4cf12a2e8'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:03.876709 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-d1f0fe3e-05ad-4db1-a44b-da41bbb7034f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a317c1c88d449ca9c7109b4cf12a2e8'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:03.883710 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-d1f0fe3e-05ad-4db1-a44b-da41bbb7034f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a317c1c88d449ca9c7109b4cf12a2e8'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:03.896711 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-d1f0fe3e-05ad-4db1-a44b-da41bbb7034f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a317c1c88d449ca9c7109b4cf12a2e8'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:03.902827 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1440/2855] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 13:02:03 2022] DELETE /identity/v3/users/6a317c1c88d449ca9c7109b4cf12a2e8 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:02:04.173427 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef8601-a13a-4802-8fbd-8ce3ace65681 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:04.174961 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-c0ef8601-a13a-4802-8fbd-8ce3ace65681 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:02:04.193494 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef8601-a13a-4802-8fbd-8ce3ace65681 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:04.250530 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae928cf-dc86-407c-99e2-420079330c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:04.251092 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae928cf-dc86-407c-99e2-420079330c73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:04.251369 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae928cf-dc86-407c-99e2-420079330c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:04.251669 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae928cf-dc86-407c-99e2-420079330c73 admin admin] PATH_INFO: `/v3/projects/a6f59bc5df02426983e0e8fe2a4bfa93` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:04.252840 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae928cf-dc86-407c-99e2-420079330c73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6f59bc5df02426983e0e8fe2a4bfa93)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:04.254747 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae928cf-dc86-407c-99e2-420079330c73 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:04.293400 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:02:04.302471 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-dae928cf-dc86-407c-99e2-420079330c73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6f59bc5df02426983e0e8fe2a4bfa93', 'request_id': 'req-dae928cf-dc86-407c-99e2-420079330c73'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:04.308744 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-dae928cf-dc86-407c-99e2-420079330c73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6f59bc5df02426983e0e8fe2a4bfa93', 'request_id': 'req-dae928cf-dc86-407c-99e2-420079330c73'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:04.316721 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-dae928cf-dc86-407c-99e2-420079330c73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6f59bc5df02426983e0e8fe2a4bfa93', 'request_id': 'req-dae928cf-dc86-407c-99e2-420079330c73'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:04.326986 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-dae928cf-dc86-407c-99e2-420079330c73 admin admin] The token cache is being invalidate because project a6f59bc5df02426983e0e8fe2a4bfa93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:02:04.327271 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-dae928cf-dc86-407c-99e2-420079330c73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:04.327929 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-dae928cf-dc86-407c-99e2-420079330c73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:04.329464 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1416/2856] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 13:02:04 2022] DELETE /identity/v3/projects/a6f59bc5df02426983e0e8fe2a4bfa93 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:02:04.334543 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7abf1a-e6d3-43ca-a182-8c7d605b97f1 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:04.335016 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7abf1a-e6d3-43ca-a182-8c7d605b97f1 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:04.335145 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7abf1a-e6d3-43ca-a182-8c7d605b97f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:04.354483 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-3c7abf1a-e6d3-43ca-a182-8c7d605b97f1 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:02:04.355717 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-3c7abf1a-e6d3-43ca-a182-8c7d605b97f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:02:04.364590 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:02:04.417789 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1441/2857] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 13:02:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:02:04.425700 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090fbeda-079a-4643-a423-a1b31d3fd592 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:04.425986 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090fbeda-079a-4643-a423-a1b31d3fd592 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:04.426262 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090fbeda-079a-4643-a423-a1b31d3fd592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:04.447061 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-090fbeda-079a-4643-a423-a1b31d3fd592 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:02:04.448360 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-090fbeda-079a-4643-a423-a1b31d3fd592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:02:04.456741 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1417/2858] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 13:02:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:02:04.461040 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7abf1a-e6d3-43ca-a182-8c7d605b97f1 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:04.471412 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7abf1a-e6d3-43ca-a182-8c7d605b97f1 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:04.476186 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1efea6f-6022-42ff-8aa4-45e97d5d1d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:04.476797 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1efea6f-6022-42ff-8aa4-45e97d5d1d6b admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:04.477112 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1efea6f-6022-42ff-8aa4-45e97d5d1d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:04.477388 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1efea6f-6022-42ff-8aa4-45e97d5d1d6b admin admin] PATH_INFO: `/v3/domains` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:04.478091 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1efea6f-6022-42ff-8aa4-45e97d5d1d6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:02:04.478396 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1efea6f-6022-42ff-8aa4-45e97d5d1d6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:04.480032 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1efea6f-6022-42ff-8aa4-45e97d5d1d6b admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:04.493126 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1442/2859] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Jul 26 13:02:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:02:04.497422 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090fbeda-079a-4643-a423-a1b31d3fd592 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:04.507521 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090fbeda-079a-4643-a423-a1b31d3fd592 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:04.512574 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee8fc6-5f07-461e-b404-585b828b6ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:04.513139 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee8fc6-5f07-461e-b404-585b828b6ad2 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:04.513456 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee8fc6-5f07-461e-b404-585b828b6ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:04.513684 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee8fc6-5f07-461e-b404-585b828b6ad2 admin admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:04.514943 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ee8fc6-5f07-461e-b404-585b828b6ad2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:04.516861 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ee8fc6-5f07-461e-b404-585b828b6ad2 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:04.547313 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1418/2860] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 13:02:04 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:02:04.548492 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1efea6f-6022-42ff-8aa4-45e97d5d1d6b admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:04.558640 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1efea6f-6022-42ff-8aa4-45e97d5d1d6b admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:04.619054 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2d312-7723-4412-82f6-38ce8fc29ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:04.619842 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2d312-7723-4412-82f6-38ce8fc29ae7 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:04.620123 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2d312-7723-4412-82f6-38ce8fc29ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:04.620399 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2d312-7723-4412-82f6-38ce8fc29ae7 admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:04.621073 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d2d312-7723-4412-82f6-38ce8fc29ae7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:04.623015 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d2d312-7723-4412-82f6-38ce8fc29ae7 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:04.648454 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1443/2861] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 13:02:04 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:02:04.652452 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee8fc6-5f07-461e-b404-585b828b6ad2 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:04.662641 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee8fc6-5f07-461e-b404-585b828b6ad2 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:04.667250 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9175f24f-659e-463d-a7eb-8543e73a2d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:04.667950 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175f24f-659e-463d-a7eb-8543e73a2d31 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:04.668361 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175f24f-659e-463d-a7eb-8543e73a2d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:04.668466 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175f24f-659e-463d-a7eb-8543e73a2d31 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:04.669152 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9175f24f-659e-463d-a7eb-8543e73a2d31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:04.670740 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9175f24f-659e-463d-a7eb-8543e73a2d31 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:04.682879 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1419/2862] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 13:02:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 13:02:04.687478 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2d312-7723-4412-82f6-38ce8fc29ae7 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:04.698190 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2d312-7723-4412-82f6-38ce8fc29ae7 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:04.703520 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd346741-b2bf-40fc-b69a-1ec50e3c85cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:04.704171 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd346741-b2bf-40fc-b69a-1ec50e3c85cd admin admin] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:04.704454 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd346741-b2bf-40fc-b69a-1ec50e3c85cd admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:04.704727 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd346741-b2bf-40fc-b69a-1ec50e3c85cd admin admin] PATH_INFO: `/v3/roles` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:04.705431 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd346741-b2bf-40fc-b69a-1ec50e3c85cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:04.707044 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd346741-b2bf-40fc-b69a-1ec50e3c85cd admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:04.722510 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1444/2863] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 13:02:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:02:04.724021 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9175f24f-659e-463d-a7eb-8543e73a2d31 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:04.734728 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9175f24f-659e-463d-a7eb-8543e73a2d31 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:04.739708 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37699aca-728a-47b7-9e4b-d21bdd9097ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:04.741671 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37699aca-728a-47b7-9e4b-d21bdd9097ae admin admin] REQUEST_METHOD: `PUT` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:04.741671 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37699aca-728a-47b7-9e4b-d21bdd9097ae admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:04.741671 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37699aca-728a-47b7-9e4b-d21bdd9097ae admin admin] PATH_INFO: `/v3/projects/07855345b6e741a2ae229da7a1d10b23/users/6a580d57bf7c4f8dadeee1895d71e763/roles/01c9179cca2747a6b95b0decfca8b7ae` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:04.756677 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37699aca-728a-47b7-9e4b-d21bdd9097ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=07855345b6e741a2ae229da7a1d10b23, user_id=6a580d57bf7c4f8dadeee1895d71e763, role_id=01c9179cca2747a6b95b0decfca8b7ae)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:04.759682 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37699aca-728a-47b7-9e4b-d21bdd9097ae admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:04.774323 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1420/2864] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 13:02:04 2022] PUT /identity/v3/projects/07855345b6e741a2ae229da7a1d10b23/users/6a580d57bf7c4f8dadeee1895d71e763/roles/01c9179cca2747a6b95b0decfca8b7ae => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:02:04.785650 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a326bb-d861-473e-a875-539b6fffc333 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:04.785941 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a326bb-d861-473e-a875-539b6fffc333 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:04.786244 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a326bb-d861-473e-a875-539b6fffc333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:04.814608 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-60a326bb-d861-473e-a875-539b6fffc333 None None] MFA Rules not processed for user `6a580d57bf7c4f8dadeee1895d71e763`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:02:04.815877 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-60a326bb-d861-473e-a875-539b6fffc333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:02:04.882540 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1445/2865] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 13:02:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:02:04.887011 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37699aca-728a-47b7-9e4b-d21bdd9097ae admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:04.959326 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37699aca-728a-47b7-9e4b-d21bdd9097ae admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:04.964806 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186986f7-bd0e-4426-a8bc-bc1957b9b603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:04.965553 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186986f7-bd0e-4426-a8bc-bc1957b9b603 admin admin] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:04.965902 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186986f7-bd0e-4426-a8bc-bc1957b9b603 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:04.966279 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186986f7-bd0e-4426-a8bc-bc1957b9b603 admin admin] PATH_INFO: `/v3/projects` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:04.967467 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186986f7-bd0e-4426-a8bc-bc1957b9b603 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:04.969415 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186986f7-bd0e-4426-a8bc-bc1957b9b603 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:05.005153 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1421/2866] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Jul 26 13:02:04 2022] POST /identity/v3/projects => generated 381 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:02:05.006462 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a326bb-d861-473e-a875-539b6fffc333 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:05.020094 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a326bb-d861-473e-a875-539b6fffc333 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:05.025101 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:02:05.084413 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352c26c1-b4eb-493d-a505-9bb76f273f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:05.085040 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c26c1-b4eb-493d-a505-9bb76f273f08 admin admin] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:05.085323 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c26c1-b4eb-493d-a505-9bb76f273f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:05.085595 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c26c1-b4eb-493d-a505-9bb76f273f08 admin admin] PATH_INFO: `/v3/users` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:05.086261 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352c26c1-b4eb-493d-a505-9bb76f273f08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:05.088219 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352c26c1-b4eb-493d-a505-9bb76f273f08 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:05.113719 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1446/2867] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 26 13:02:05 2022] POST /identity/v3/users => generated 297 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:02:05.116653 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186986f7-bd0e-4426-a8bc-bc1957b9b603 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:05.128613 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186986f7-bd0e-4426-a8bc-bc1957b9b603 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:05.133304 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa89e710-4350-4760-ab18-afd834131f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:05.133916 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa89e710-4350-4760-ab18-afd834131f43 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:05.134197 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa89e710-4350-4760-ab18-afd834131f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:05.134756 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa89e710-4350-4760-ab18-afd834131f43 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:05.135462 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa89e710-4350-4760-ab18-afd834131f43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:05.137111 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa89e710-4350-4760-ab18-afd834131f43 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:05.148276 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1422/2868] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 13:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:02:05.153022 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352c26c1-b4eb-493d-a505-9bb76f273f08 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:05.168785 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352c26c1-b4eb-493d-a505-9bb76f273f08 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:05.175055 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9fe752-a4e0-400b-9d08-d220111a33cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:05.175770 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9fe752-a4e0-400b-9d08-d220111a33cb admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:05.176057 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9fe752-a4e0-400b-9d08-d220111a33cb admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:05.176340 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9fe752-a4e0-400b-9d08-d220111a33cb admin admin] PATH_INFO: `/v3/projects/c746fc924eed40ae80a4b057c82a11b7/users/9eea59c436d84e8b85bdc851e529d2d2/roles/0e78444e43c744668178971af4087ca0` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:05.193526 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9fe752-a4e0-400b-9d08-d220111a33cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c746fc924eed40ae80a4b057c82a11b7, user_id=9eea59c436d84e8b85bdc851e529d2d2, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:05.196274 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9fe752-a4e0-400b-9d08-d220111a33cb admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:05.209471 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1447/2869] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 13:02:05 2022] PUT /identity/v3/projects/c746fc924eed40ae80a4b057c82a11b7/users/9eea59c436d84e8b85bdc851e529d2d2/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:02:05.213412 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa89e710-4350-4760-ab18-afd834131f43 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:05.224152 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa89e710-4350-4760-ab18-afd834131f43 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:05.260837 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:02:05.289085 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede114a2-72db-40f9-b2fd-09ef459eab79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:05.289713 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede114a2-72db-40f9-b2fd-09ef459eab79 admin admin] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:05.289990 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede114a2-72db-40f9-b2fd-09ef459eab79 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:05.290258 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede114a2-72db-40f9-b2fd-09ef459eab79 admin admin] PATH_INFO: `/v3/roles` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:05.290996 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede114a2-72db-40f9-b2fd-09ef459eab79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:05.292629 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede114a2-72db-40f9-b2fd-09ef459eab79 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:05.304273 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1423/2870] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Jul 26 13:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:02:05.308535 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9fe752-a4e0-400b-9d08-d220111a33cb admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:05.319202 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9fe752-a4e0-400b-9d08-d220111a33cb admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:05.323984 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d50a182-2897-46b7-8ca6-2da4554a352c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:05.324670 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50a182-2897-46b7-8ca6-2da4554a352c admin admin] REQUEST_METHOD: `PUT` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:05.324949 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50a182-2897-46b7-8ca6-2da4554a352c admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:05.325223 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50a182-2897-46b7-8ca6-2da4554a352c admin admin] PATH_INFO: `/v3/projects/c746fc924eed40ae80a4b057c82a11b7/users/9eea59c436d84e8b85bdc851e529d2d2/roles/0e78444e43c744668178971af4087ca0` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:05.327301 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d50a182-2897-46b7-8ca6-2da4554a352c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c746fc924eed40ae80a4b057c82a11b7, user_id=9eea59c436d84e8b85bdc851e529d2d2, role_id=0e78444e43c744668178971af4087ca0)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:05.330072 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d50a182-2897-46b7-8ca6-2da4554a352c admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:05.341911 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1448/2871] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Jul 26 13:02:05 2022] PUT /identity/v3/projects/c746fc924eed40ae80a4b057c82a11b7/users/9eea59c436d84e8b85bdc851e529d2d2/roles/0e78444e43c744668178971af4087ca0 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:02:05.352910 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a33aa6-8ab5-44c2-93a1-e96848cfc66c None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:05.353198 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a33aa6-8ab5-44c2-93a1-e96848cfc66c None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:05.353480 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a33aa6-8ab5-44c2-93a1-e96848cfc66c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:05.383981 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-63a33aa6-8ab5-44c2-93a1-e96848cfc66c None None] MFA Rules not processed for user `9eea59c436d84e8b85bdc851e529d2d2`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:02:05.385477 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-63a33aa6-8ab5-44c2-93a1-e96848cfc66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:02:05.459682 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1424/2872] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 13:02:05 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:02:05.471989 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d50a182-2897-46b7-8ca6-2da4554a352c admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:05.473751 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-6d50a182-2897-46b7-8ca6-2da4554a352c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:02:05.536613 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d50a182-2897-46b7-8ca6-2da4554a352c admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:05.541261 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda9997-5e90-4a9c-a993-023a6d77cb44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:05.541615 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda9997-5e90-4a9c-a993-023a6d77cb44 service cinder] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:05.541915 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda9997-5e90-4a9c-a993-023a6d77cb44 service cinder] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:05.542169 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda9997-5e90-4a9c-a993-023a6d77cb44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:05.555442 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcda9997-5e90-4a9c-a993-023a6d77cb44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:05.556934 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcda9997-5e90-4a9c-a993-023a6d77cb44 service cinder] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:05.567125 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1449/2873] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 13:02:05 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:02:05.642342 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a33aa6-8ab5-44c2-93a1-e96848cfc66c None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:05.653458 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a33aa6-8ab5-44c2-93a1-e96848cfc66c None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:05.657850 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11576a95-8dd8-4786-9be7-06e4d303788b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f8ca5e019dd4b9090cee2b2c1d8e15d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:05.658461 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11576a95-8dd8-4786-9be7-06e4d303788b service cinder] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:05.658747 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11576a95-8dd8-4786-9be7-06e4d303788b service cinder] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:05.659020 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11576a95-8dd8-4786-9be7-06e4d303788b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:05.738946 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11576a95-8dd8-4786-9be7-06e4d303788b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:05.740550 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11576a95-8dd8-4786-9be7-06e4d303788b service cinder] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:05.750597 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1425/2874] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 13:02:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:02:06.085637 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda9997-5e90-4a9c-a993-023a6d77cb44 service cinder] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:06.152542 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda9997-5e90-4a9c-a993-023a6d77cb44 service cinder] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:06.157737 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17d998-03e8-4d80-ba56-e95842a7b832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:06.158386 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17d998-03e8-4d80-ba56-e95842a7b832 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:06.158668 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17d998-03e8-4d80-ba56-e95842a7b832 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:06.158945 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17d998-03e8-4d80-ba56-e95842a7b832 admin admin] PATH_INFO: `/v3/users/6a580d57bf7c4f8dadeee1895d71e763` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:06.160159 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af17d998-03e8-4d80-ba56-e95842a7b832 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a580d57bf7c4f8dadeee1895d71e763)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:06.162112 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af17d998-03e8-4d80-ba56-e95842a7b832 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:06.167213 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:02:06.214627 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-af17d998-03e8-4d80-ba56-e95842a7b832 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a580d57bf7c4f8dadeee1895d71e763'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:06.222715 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-af17d998-03e8-4d80-ba56-e95842a7b832 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a580d57bf7c4f8dadeee1895d71e763'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:06.223407 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-af17d998-03e8-4d80-ba56-e95842a7b832 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a580d57bf7c4f8dadeee1895d71e763'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:06.230811 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-af17d998-03e8-4d80-ba56-e95842a7b832 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a580d57bf7c4f8dadeee1895d71e763'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:06.244554 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-af17d998-03e8-4d80-ba56-e95842a7b832 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a580d57bf7c4f8dadeee1895d71e763'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:06.251600 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1450/2875] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 13:02:06 2022] DELETE /identity/v3/users/6a580d57bf7c4f8dadeee1895d71e763 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:02:06.256109 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae99f3-7eba-40f8-909d-8a207ca11619 None None] REQUEST_METHOD: `POST` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:06.256622 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae99f3-7eba-40f8-909d-8a207ca11619 None None] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:06.256774 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae99f3-7eba-40f8-909d-8a207ca11619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:06.276226 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.auth.core [None req-dbae99f3-7eba-40f8-909d-8a207ca11619 None None] MFA Rules not processed for user `26e313f5057b4411a71a6bd67e68cb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:02:06.277486 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-dbae99f3-7eba-40f8-909d-8a207ca11619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:02:06.336287 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=100 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:02:06.347036 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1426/2876] 10.222.0.20 () {58 vars in 958 bytes} [Tue Jul 26 13:02:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 13:02:06.356731 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17d998-03e8-4d80-ba56-e95842a7b832 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:06.358262 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-af17d998-03e8-4d80-ba56-e95842a7b832 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:02:06.416756 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17d998-03e8-4d80-ba56-e95842a7b832 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:06.420510 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840408fc-6d98-4e0c-91e3-03c4935ae8c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:06.421181 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840408fc-6d98-4e0c-91e3-03c4935ae8c4 service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:06.421464 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840408fc-6d98-4e0c-91e3-03c4935ae8c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:06.421866 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840408fc-6d98-4e0c-91e3-03c4935ae8c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:06.431672 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840408fc-6d98-4e0c-91e3-03c4935ae8c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:06.433327 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840408fc-6d98-4e0c-91e3-03c4935ae8c4 service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:06.444994 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1451/2877] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 13:02:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:02:06.745298 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbae99f3-7eba-40f8-909d-8a207ca11619 None None] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:06.747444 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-dbae99f3-7eba-40f8-909d-8a207ca11619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:02:06.762406 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbae99f3-7eba-40f8-909d-8a207ca11619 None None] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:06.768565 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555cd35d-c28f-4643-9541-49505deb7faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:06.769395 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555cd35d-c28f-4643-9541-49505deb7faa admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:06.769673 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555cd35d-c28f-4643-9541-49505deb7faa admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:06.769949 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555cd35d-c28f-4643-9541-49505deb7faa admin admin] PATH_INFO: `/v3/projects/07855345b6e741a2ae229da7a1d10b23` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:06.771307 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555cd35d-c28f-4643-9541-49505deb7faa admin admin] RBAC: Authorizing `identity:delete_project(project_id=07855345b6e741a2ae229da7a1d10b23)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:06.774481 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555cd35d-c28f-4643-9541-49505deb7faa admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:06.822830 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-555cd35d-c28f-4643-9541-49505deb7faa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07855345b6e741a2ae229da7a1d10b23', 'request_id': 'req-555cd35d-c28f-4643-9541-49505deb7faa'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:06.829625 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-555cd35d-c28f-4643-9541-49505deb7faa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07855345b6e741a2ae229da7a1d10b23', 'request_id': 'req-555cd35d-c28f-4643-9541-49505deb7faa'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:06.837437 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-555cd35d-c28f-4643-9541-49505deb7faa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07855345b6e741a2ae229da7a1d10b23', 'request_id': 'req-555cd35d-c28f-4643-9541-49505deb7faa'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:06.848136 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-555cd35d-c28f-4643-9541-49505deb7faa admin admin] The token cache is being invalidate because project 07855345b6e741a2ae229da7a1d10b23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:02:06.848358 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-555cd35d-c28f-4643-9541-49505deb7faa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:06.848969 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-555cd35d-c28f-4643-9541-49505deb7faa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:06.850471 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1427/2878] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 13:02:06 2022] DELETE /identity/v3/projects/07855345b6e741a2ae229da7a1d10b23 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:02:06.854444 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840408fc-6d98-4e0c-91e3-03c4935ae8c4 service neutron] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:06.864005 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840408fc-6d98-4e0c-91e3-03c4935ae8c4 service neutron] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:06.921521 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59c09c6-609b-4400-9f6f-552db87f1a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:06.922156 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c09c6-609b-4400-9f6f-552db87f1a91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:06.922461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c09c6-609b-4400-9f6f-552db87f1a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:06.922738 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c09c6-609b-4400-9f6f-552db87f1a91 admin admin] PATH_INFO: `/v3/users/9eea59c436d84e8b85bdc851e529d2d2` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:06.923917 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59c09c6-609b-4400-9f6f-552db87f1a91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9eea59c436d84e8b85bdc851e529d2d2)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:06.925847 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59c09c6-609b-4400-9f6f-552db87f1a91 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:06.955174 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:02:06.976750 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-f59c09c6-609b-4400-9f6f-552db87f1a91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9eea59c436d84e8b85bdc851e529d2d2'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:06.983963 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-f59c09c6-609b-4400-9f6f-552db87f1a91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9eea59c436d84e8b85bdc851e529d2d2'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:06.984612 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-f59c09c6-609b-4400-9f6f-552db87f1a91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9eea59c436d84e8b85bdc851e529d2d2'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:06.993349 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-f59c09c6-609b-4400-9f6f-552db87f1a91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9eea59c436d84e8b85bdc851e529d2d2'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:07.007180 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-f59c09c6-609b-4400-9f6f-552db87f1a91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9eea59c436d84e8b85bdc851e529d2d2'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:07.014493 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1452/2879] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 13:02:06 2022] DELETE /identity/v3/users/9eea59c436d84e8b85bdc851e529d2d2 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:02:07.396012 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555cd35d-c28f-4643-9541-49505deb7faa admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:07.398252 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-555cd35d-c28f-4643-9541-49505deb7faa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:02:07.427813 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555cd35d-c28f-4643-9541-49505deb7faa admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:07.493456 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65f04d3-90de-4ef9-b2f1-77f2bfb864ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:07.494084 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f04d3-90de-4ef9-b2f1-77f2bfb864ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:07.494455 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f04d3-90de-4ef9-b2f1-77f2bfb864ad admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:07.494617 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f04d3-90de-4ef9-b2f1-77f2bfb864ad admin admin] PATH_INFO: `/v3/projects/c746fc924eed40ae80a4b057c82a11b7` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:07.495795 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65f04d3-90de-4ef9-b2f1-77f2bfb864ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=c746fc924eed40ae80a4b057c82a11b7)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:07.497693 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65f04d3-90de-4ef9-b2f1-77f2bfb864ad admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:07.546143 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-c65f04d3-90de-4ef9-b2f1-77f2bfb864ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c746fc924eed40ae80a4b057c82a11b7', 'request_id': 'req-c65f04d3-90de-4ef9-b2f1-77f2bfb864ad'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:07.553073 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-c65f04d3-90de-4ef9-b2f1-77f2bfb864ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c746fc924eed40ae80a4b057c82a11b7', 'request_id': 'req-c65f04d3-90de-4ef9-b2f1-77f2bfb864ad'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:07.561183 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-c65f04d3-90de-4ef9-b2f1-77f2bfb864ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c746fc924eed40ae80a4b057c82a11b7', 'request_id': 'req-c65f04d3-90de-4ef9-b2f1-77f2bfb864ad'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:07.571847 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-c65f04d3-90de-4ef9-b2f1-77f2bfb864ad admin admin] The token cache is being invalidate because project c746fc924eed40ae80a4b057c82a11b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:02:07.572361 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-c65f04d3-90de-4ef9-b2f1-77f2bfb864ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:07.573171 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-c65f04d3-90de-4ef9-b2f1-77f2bfb864ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:07.575102 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1428/2880] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 13:02:07 2022] DELETE /identity/v3/projects/c746fc924eed40ae80a4b057c82a11b7 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:02:11.514225 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59c09c6-609b-4400-9f6f-552db87f1a91 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:11.515947 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-f59c09c6-609b-4400-9f6f-552db87f1a91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:02:11.582888 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59c09c6-609b-4400-9f6f-552db87f1a91 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:11.637695 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750f73f9-97e8-4efc-8b19-7093740e4470 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6830350c6eba4610bbe9ab1dc31f5939', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:11.638323 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f73f9-97e8-4efc-8b19-7093740e4470 service neutron] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:11.639245 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f73f9-97e8-4efc-8b19-7093740e4470 service neutron] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:11.639459 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f73f9-97e8-4efc-8b19-7093740e4470 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:11.645123 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-750f73f9-97e8-4efc-8b19-7093740e4470 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:02:11.733876 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750f73f9-97e8-4efc-8b19-7093740e4470 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:11.735323 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750f73f9-97e8-4efc-8b19-7093740e4470 service neutron] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:11.745610 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1453/2881] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 13:02:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:02:15.145826 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65f04d3-90de-4ef9-b2f1-77f2bfb864ad admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:15.149155 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-c65f04d3-90de-4ef9-b2f1-77f2bfb864ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:02:15.166897 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:02:15.245591 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65f04d3-90de-4ef9-b2f1-77f2bfb864ad admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:15.251090 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec067d2-5d26-446a-b2aa-d55f40a1aa87 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec49a03aed774c439459883bd5442d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:15.251736 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec067d2-5d26-446a-b2aa-d55f40a1aa87 service placement] REQUEST_METHOD: `GET` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:15.252020 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec067d2-5d26-446a-b2aa-d55f40a1aa87 service placement] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:15.252293 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec067d2-5d26-446a-b2aa-d55f40a1aa87 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:15.254202 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-6ec067d2-5d26-446a-b2aa-d55f40a1aa87 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:02:15.267523 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec067d2-5d26-446a-b2aa-d55f40a1aa87 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:15.269145 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec067d2-5d26-446a-b2aa-d55f40a1aa87 service placement] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:15.279487 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1429/2882] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 13:02:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:02:18.495030 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87702581-2758-4782-a704-aa9c27f8b661 None None] REQUEST_METHOD: `POST` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:18.495612 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87702581-2758-4782-a704-aa9c27f8b661 None None] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:18.496190 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87702581-2758-4782-a704-aa9c27f8b661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:18.520881 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:02:18.572312 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.auth.core [None req-87702581-2758-4782-a704-aa9c27f8b661 None None] MFA Rules not processed for user `30fc532f464843a3ad35e4d3c387bb3a`. Rule list: `[]` (Enabled: `True`). {{(pid=91749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:02:18.574061 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-87702581-2758-4782-a704-aa9c27f8b661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:02:18.603683 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec067d2-5d26-446a-b2aa-d55f40a1aa87 service placement] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:18.606100 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-6ec067d2-5d26-446a-b2aa-d55f40a1aa87 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:02:18.631350 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec067d2-5d26-446a-b2aa-d55f40a1aa87 service placement] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:18.648860 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60711e5e-2315-4848-97bd-b68ea1859bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:18.649603 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60711e5e-2315-4848-97bd-b68ea1859bac admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:18.650051 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60711e5e-2315-4848-97bd-b68ea1859bac admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:18.650533 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60711e5e-2315-4848-97bd-b68ea1859bac admin admin] PATH_INFO: `/v3/users/94efba410653451da2e9977d746633f6` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:18.651979 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60711e5e-2315-4848-97bd-b68ea1859bac admin admin] RBAC: Authorizing `identity:delete_user(user_id=94efba410653451da2e9977d746633f6)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:18.654087 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60711e5e-2315-4848-97bd-b68ea1859bac admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:18.665744 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1454/2883] 10.222.0.20 () {58 vars in 967 bytes} [Tue Jul 26 13:02:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 13:02:18.706853 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-60711e5e-2315-4848-97bd-b68ea1859bac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94efba410653451da2e9977d746633f6'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:18.707714 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-60711e5e-2315-4848-97bd-b68ea1859bac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94efba410653451da2e9977d746633f6'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:18.708405 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-60711e5e-2315-4848-97bd-b68ea1859bac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94efba410653451da2e9977d746633f6'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:18.719233 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87702581-2758-4782-a704-aa9c27f8b661 None None] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:18.720603 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-87702581-2758-4782-a704-aa9c27f8b661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:02:18.723055 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-60711e5e-2315-4848-97bd-b68ea1859bac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94efba410653451da2e9977d746633f6'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:18.731737 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-60711e5e-2315-4848-97bd-b68ea1859bac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94efba410653451da2e9977d746633f6'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:18.746566 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1430/2884] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 13:02:18 2022] DELETE /identity/v3/users/94efba410653451da2e9977d746633f6 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:02:18.816775 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87702581-2758-4782-a704-aa9c27f8b661 None None] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:18.881374 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e409b20-c2e0-4e20-be5f-7401f44c34ea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd2483f74d7d48b1a7b8ffe32f63abde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:18.882018 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e409b20-c2e0-4e20-be5f-7401f44c34ea service swift] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:18.882302 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e409b20-c2e0-4e20-be5f-7401f44c34ea service swift] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:18.882575 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e409b20-c2e0-4e20-be5f-7401f44c34ea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:18.884728 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-6e409b20-c2e0-4e20-be5f-7401f44c34ea service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:02:18.890903 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=100 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:02:18.933076 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60711e5e-2315-4848-97bd-b68ea1859bac admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:18.934679 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-60711e5e-2315-4848-97bd-b68ea1859bac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:02:18.950656 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e409b20-c2e0-4e20-be5f-7401f44c34ea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:18.952527 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e409b20-c2e0-4e20-be5f-7401f44c34ea service swift] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:18.968990 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1455/2885] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Jul 26 13:02:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 26 13:02:19.016164 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60711e5e-2315-4848-97bd-b68ea1859bac admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:19.020944 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c5a31-5fd0-44d4-b2f3-f59cfaafb8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:19.021610 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c5a31-5fd0-44d4-b2f3-f59cfaafb8d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:19.021854 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c5a31-5fd0-44d4-b2f3-f59cfaafb8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:19.022122 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c5a31-5fd0-44d4-b2f3-f59cfaafb8d8 admin admin] PATH_INFO: `/v3/projects/a2522985c6df452dabe2010e5f6bed35` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:19.023348 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60c5a31-5fd0-44d4-b2f3-f59cfaafb8d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2522985c6df452dabe2010e5f6bed35)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:19.025316 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60c5a31-5fd0-44d4-b2f3-f59cfaafb8d8 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:19.048996 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:02:19.092852 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-d60c5a31-5fd0-44d4-b2f3-f59cfaafb8d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2522985c6df452dabe2010e5f6bed35', 'request_id': 'req-d60c5a31-5fd0-44d4-b2f3-f59cfaafb8d8'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:19.099302 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-d60c5a31-5fd0-44d4-b2f3-f59cfaafb8d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2522985c6df452dabe2010e5f6bed35', 'request_id': 'req-d60c5a31-5fd0-44d4-b2f3-f59cfaafb8d8'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:19.109056 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-d60c5a31-5fd0-44d4-b2f3-f59cfaafb8d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2522985c6df452dabe2010e5f6bed35', 'request_id': 'req-d60c5a31-5fd0-44d4-b2f3-f59cfaafb8d8'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:19.120626 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-d60c5a31-5fd0-44d4-b2f3-f59cfaafb8d8 admin admin] The token cache is being invalidate because project a2522985c6df452dabe2010e5f6bed35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:02:19.120919 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-d60c5a31-5fd0-44d4-b2f3-f59cfaafb8d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:19.123398 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-d60c5a31-5fd0-44d4-b2f3-f59cfaafb8d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:19.129431 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e409b20-c2e0-4e20-be5f-7401f44c34ea service swift] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:19.129579 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1431/2886] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 13:02:18 2022] DELETE /identity/v3/projects/a2522985c6df452dabe2010e5f6bed35 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 13:02:19.140619 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e409b20-c2e0-4e20-be5f-7401f44c34ea service swift] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:19.200101 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c90312-71bd-4b29-9fd0-eff91340a704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:19.200667 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c90312-71bd-4b29-9fd0-eff91340a704 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:19.201468 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c90312-71bd-4b29-9fd0-eff91340a704 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:19.201746 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c90312-71bd-4b29-9fd0-eff91340a704 admin admin] PATH_INFO: `/v3/users/82223cea11be4790a233d1419c5e37b9` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:19.202939 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c90312-71bd-4b29-9fd0-eff91340a704 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82223cea11be4790a233d1419c5e37b9)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:19.205114 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c90312-71bd-4b29-9fd0-eff91340a704 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:19.256534 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-91c90312-71bd-4b29-9fd0-eff91340a704 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82223cea11be4790a233d1419c5e37b9'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:19.264799 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-91c90312-71bd-4b29-9fd0-eff91340a704 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82223cea11be4790a233d1419c5e37b9'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:19.265419 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-91c90312-71bd-4b29-9fd0-eff91340a704 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82223cea11be4790a233d1419c5e37b9'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:19.272408 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-91c90312-71bd-4b29-9fd0-eff91340a704 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82223cea11be4790a233d1419c5e37b9'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:19.285992 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-91c90312-71bd-4b29-9fd0-eff91340a704 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82223cea11be4790a233d1419c5e37b9'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:19.292169 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1456/2887] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 13:02:19 2022] DELETE /identity/v3/users/82223cea11be4790a233d1419c5e37b9 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:02:19.477547 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c5a31-5fd0-44d4-b2f3-f59cfaafb8d8 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:19.479149 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-d60c5a31-5fd0-44d4-b2f3-f59cfaafb8d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:02:19.497391 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c5a31-5fd0-44d4-b2f3-f59cfaafb8d8 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:19.571268 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc5d50c-2955-4e52-83a6-018640adce07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:19.571974 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5d50c-2955-4e52-83a6-018640adce07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:19.572257 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5d50c-2955-4e52-83a6-018640adce07 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:19.572538 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5d50c-2955-4e52-83a6-018640adce07 admin admin] PATH_INFO: `/v3/projects/3b53d8ebad9843729c9d9d8c5a83c2db` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:19.574655 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc5d50c-2955-4e52-83a6-018640adce07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b53d8ebad9843729c9d9d8c5a83c2db)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:19.576534 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc5d50c-2955-4e52-83a6-018640adce07 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:19.641943 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-3fc5d50c-2955-4e52-83a6-018640adce07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b53d8ebad9843729c9d9d8c5a83c2db', 'request_id': 'req-3fc5d50c-2955-4e52-83a6-018640adce07'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:19.650427 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-3fc5d50c-2955-4e52-83a6-018640adce07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b53d8ebad9843729c9d9d8c5a83c2db', 'request_id': 'req-3fc5d50c-2955-4e52-83a6-018640adce07'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:19.662564 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-3fc5d50c-2955-4e52-83a6-018640adce07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b53d8ebad9843729c9d9d8c5a83c2db', 'request_id': 'req-3fc5d50c-2955-4e52-83a6-018640adce07'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:19.682139 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-3fc5d50c-2955-4e52-83a6-018640adce07 admin admin] The token cache is being invalidate because project 3b53d8ebad9843729c9d9d8c5a83c2db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:02:19.682380 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-3fc5d50c-2955-4e52-83a6-018640adce07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:19.683137 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-3fc5d50c-2955-4e52-83a6-018640adce07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:19.687853 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1432/2888] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 13:02:19 2022] DELETE /identity/v3/projects/3b53d8ebad9843729c9d9d8c5a83c2db => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:02:25.726264 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c90312-71bd-4b29-9fd0-eff91340a704 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:25.727836 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-91c90312-71bd-4b29-9fd0-eff91340a704 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:02:25.744072 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c90312-71bd-4b29-9fd0-eff91340a704 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:25.775553 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:02:25.809616 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d454d7-536e-4789-a437-e79ad213ae75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:25.810774 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d454d7-536e-4789-a437-e79ad213ae75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:25.811057 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d454d7-536e-4789-a437-e79ad213ae75 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:25.811333 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d454d7-536e-4789-a437-e79ad213ae75 admin admin] PATH_INFO: `/v3/users/68818cd15fad466cb7cb5584212fe2f9` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:25.812546 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d454d7-536e-4789-a437-e79ad213ae75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68818cd15fad466cb7cb5584212fe2f9)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:25.814516 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d454d7-536e-4789-a437-e79ad213ae75 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:25.862074 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-96d454d7-536e-4789-a437-e79ad213ae75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68818cd15fad466cb7cb5584212fe2f9'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:25.869243 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-96d454d7-536e-4789-a437-e79ad213ae75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68818cd15fad466cb7cb5584212fe2f9'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:25.869859 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-96d454d7-536e-4789-a437-e79ad213ae75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68818cd15fad466cb7cb5584212fe2f9'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:25.877443 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-96d454d7-536e-4789-a437-e79ad213ae75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68818cd15fad466cb7cb5584212fe2f9'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:25.889542 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-96d454d7-536e-4789-a437-e79ad213ae75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68818cd15fad466cb7cb5584212fe2f9'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:25.895310 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1457/2889] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 13:02:25 2022] DELETE /identity/v3/users/68818cd15fad466cb7cb5584212fe2f9 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:02:26.081082 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc5d50c-2955-4e52-83a6-018640adce07 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:26.082758 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-3fc5d50c-2955-4e52-83a6-018640adce07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:02:26.097383 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc5d50c-2955-4e52-83a6-018640adce07 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:26.130078 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:02:26.158491 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dfb73c-9a11-452f-8ea8-aed876984201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:26.159137 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfb73c-9a11-452f-8ea8-aed876984201 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:26.159418 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfb73c-9a11-452f-8ea8-aed876984201 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:26.159723 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfb73c-9a11-452f-8ea8-aed876984201 admin admin] PATH_INFO: `/v3/projects/2ef477e2598b4c7bbd40c06ccd26112b` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:26.160896 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dfb73c-9a11-452f-8ea8-aed876984201 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ef477e2598b4c7bbd40c06ccd26112b)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:26.162818 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dfb73c-9a11-452f-8ea8-aed876984201 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:26.209477 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-71dfb73c-9a11-452f-8ea8-aed876984201 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ef477e2598b4c7bbd40c06ccd26112b', 'request_id': 'req-71dfb73c-9a11-452f-8ea8-aed876984201'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:26.215460 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-71dfb73c-9a11-452f-8ea8-aed876984201 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ef477e2598b4c7bbd40c06ccd26112b', 'request_id': 'req-71dfb73c-9a11-452f-8ea8-aed876984201'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:26.223559 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-71dfb73c-9a11-452f-8ea8-aed876984201 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ef477e2598b4c7bbd40c06ccd26112b', 'request_id': 'req-71dfb73c-9a11-452f-8ea8-aed876984201'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:26.235325 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-71dfb73c-9a11-452f-8ea8-aed876984201 admin admin] The token cache is being invalidate because project 2ef477e2598b4c7bbd40c06ccd26112b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:02:26.235591 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-71dfb73c-9a11-452f-8ea8-aed876984201 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:26.236255 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-71dfb73c-9a11-452f-8ea8-aed876984201 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:26.237964 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1433/2890] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 13:02:26 2022] DELETE /identity/v3/projects/2ef477e2598b4c7bbd40c06ccd26112b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:02:26.241709 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d454d7-536e-4789-a437-e79ad213ae75 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:26.243114 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-96d454d7-536e-4789-a437-e79ad213ae75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:02:26.255774 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d454d7-536e-4789-a437-e79ad213ae75 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:26.311398 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1fcd86-6b7e-4df1-b42a-8b264fa9bc86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:26.312056 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1fcd86-6b7e-4df1-b42a-8b264fa9bc86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:26.312335 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1fcd86-6b7e-4df1-b42a-8b264fa9bc86 admin admin] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:26.312604 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1fcd86-6b7e-4df1-b42a-8b264fa9bc86 admin admin] PATH_INFO: `/v3/users/8c94c2ead3be491889350f563a9f808b` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:26.313754 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1fcd86-6b7e-4df1-b42a-8b264fa9bc86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c94c2ead3be491889350f563a9f808b)` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:26.315688 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1fcd86-6b7e-4df1-b42a-8b264fa9bc86 admin admin] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:26.360972 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-3c1fcd86-6b7e-4df1-b42a-8b264fa9bc86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c94c2ead3be491889350f563a9f808b'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:26.362596 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:02:26.368807 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-3c1fcd86-6b7e-4df1-b42a-8b264fa9bc86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c94c2ead3be491889350f563a9f808b'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:26.369867 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-3c1fcd86-6b7e-4df1-b42a-8b264fa9bc86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c94c2ead3be491889350f563a9f808b'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:26.377101 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-3c1fcd86-6b7e-4df1-b42a-8b264fa9bc86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c94c2ead3be491889350f563a9f808b'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:26.389057 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.notifications [None req-3c1fcd86-6b7e-4df1-b42a-8b264fa9bc86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c94c2ead3be491889350f563a9f808b'} {{(pid=91749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:26.394836 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1458/2891] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Jul 26 13:02:26 2022] DELETE /identity/v3/users/8c94c2ead3be491889350f563a9f808b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:02:26.571462 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dfb73c-9a11-452f-8ea8-aed876984201 admin admin] Authenticating user token {{(pid=91748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:02:26.572920 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.fernet_utils [None req-71dfb73c-9a11-452f-8ea8-aed876984201 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:02:26.586916 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dfb73c-9a11-452f-8ea8-aed876984201 admin admin] Validating token access rules against request {{(pid=91748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:02:26.651492 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bdde78-c41d-4505-8e8d-8a0cb0de7e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26e313f5057b4411a71a6bd67e68cb0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ca671277c17458082e4383a242d5404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:02:26.652195 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bdde78-c41d-4505-8e8d-8a0cb0de7e56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:02:26.652482 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bdde78-c41d-4505-8e8d-8a0cb0de7e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:02:26.652762 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bdde78-c41d-4505-8e8d-8a0cb0de7e56 admin admin] PATH_INFO: `/v3/projects/48549592401f40a4a75d41d0a274cb8e` {{(pid=91748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:02:26.654530 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bdde78-c41d-4505-8e8d-8a0cb0de7e56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48549592401f40a4a75d41d0a274cb8e)` {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:02:26.656507 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bdde78-c41d-4505-8e8d-8a0cb0de7e56 admin admin] RBAC: Authorization granted {{(pid=91748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:02:26.703336 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-e1bdde78-c41d-4505-8e8d-8a0cb0de7e56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48549592401f40a4a75d41d0a274cb8e', 'request_id': 'req-e1bdde78-c41d-4505-8e8d-8a0cb0de7e56'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:26.709460 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-e1bdde78-c41d-4505-8e8d-8a0cb0de7e56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48549592401f40a4a75d41d0a274cb8e', 'request_id': 'req-e1bdde78-c41d-4505-8e8d-8a0cb0de7e56'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:26.717057 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-e1bdde78-c41d-4505-8e8d-8a0cb0de7e56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48549592401f40a4a75d41d0a274cb8e', 'request_id': 'req-e1bdde78-c41d-4505-8e8d-8a0cb0de7e56'} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:26.720962 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:02:26.729208 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-e1bdde78-c41d-4505-8e8d-8a0cb0de7e56 admin admin] The token cache is being invalidate because project 48549592401f40a4a75d41d0a274cb8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:02:26.729714 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-e1bdde78-c41d-4505-8e8d-8a0cb0de7e56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:26.730554 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG keystone.notifications [None req-e1bdde78-c41d-4505-8e8d-8a0cb0de7e56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:02:26.732148 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: [pid: 91748|app: 0|req: 1434/2892] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Jul 26 13:02:26 2022] DELETE /identity/v3/projects/48549592401f40a4a75d41d0a274cb8e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:02:36.391524 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=12 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:02:36.730730 dv-cinder-850939-2-iscsi devstack@keystone.service[91748]: DEBUG dbcounter [-] [91748] Writing DB stats keystone:DELETE=1,keystone:SELECT=2 {{(pid=91748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:05:28.965269 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1fcd86-6b7e-4df1-b42a-8b264fa9bc86 admin admin] Authenticating user token {{(pid=91749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:05:28.966664 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-3c1fcd86-6b7e-4df1-b42a-8b264fa9bc86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:05:29.037250 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1fcd86-6b7e-4df1-b42a-8b264fa9bc86 admin admin] Validating token access rules against request {{(pid=91749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:05:29.096424 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9526151-780a-4460-ae55-0dfecc437cc1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec49a03aed774c439459883bd5442d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '644278e7e8934b37a1e7ab20d35764cd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:05:29.097041 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9526151-780a-4460-ae55-0dfecc437cc1 service placement] REQUEST_METHOD: `GET` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:05:29.097324 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9526151-780a-4460-ae55-0dfecc437cc1 service placement] SCRIPT_NAME: `/identity` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:05:29.097598 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9526151-780a-4460-ae55-0dfecc437cc1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:05:29.099355 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.fernet_utils [None req-a9526151-780a-4460-ae55-0dfecc437cc1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:05:29.111461 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9526151-780a-4460-ae55-0dfecc437cc1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:05:29.113080 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9526151-780a-4460-ae55-0dfecc437cc1 service placement] RBAC: Authorization granted {{(pid=91749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:05:29.122933 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: [pid: 91749|app: 0|req: 1459/2893] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 26 13:05:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:05:39.118786 dv-cinder-850939-2-iscsi devstack@keystone.service[91749]: DEBUG dbcounter [-] [91749] Writing DB stats keystone:SELECT=58 {{(pid=91749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}